IIRC, there is also some change in flavor of the hops addition when there is yeast present. Again, some chemical reaction that I don't understand, but have read about. Others may be able to shed more light on this.
There is recent data (refer to Stan Heironymous's excellent "For the Love of Hops" for more details) that yeast metabolize some of the flavor and aroma compounds from hops and convert them to different substances. The specific changes depend on the yeast strain and hops. It's definitely not something you can make broad assumptions across the board. A lot of the compounds involved can be perceived in multiple ways depending on concentration, what other compounds are present in combination, as well as your own nose or palate.
Regarding the suggestion that yeast pull hop compounds out of solution, while that may be true to some extent, there is also some research showing that yeast actually increase the level of certain hop compounds. This is done by breaking down glycosides that some of these hop compounds are bound to. Again, the end result is something best judged by one's own palate.
In other words, there is no "right" way to do it. I add my dry hops right in primary while there is still a bit of yeast activity (usually day 7 of fermentation), and I am quite happy with my brews. I have yet to do a controlled test comparing a beer that has been dry-hopped in primary vs dry-hopped in secondary, but that's definitely in the plans for me.