My hop bags are utterly disgusting looking. Many, many batches of beer under their drawstrings. I toss 'em in a bowl of Star San for a few minutes, wring 'em out then add the hops, a weight and toss 'em into the fermenter or keg, where ever I want the hops. Seriously.....never a problem. I even (shame on me) neglected to clean them immediately after use one time and they molded. Scraped the mold off, washed and bleached the bags and I continue to use them.
I would never worry about contamination of a beer just through adding hops...remember, when dry hopping there's alcohol in there already and the hops are recognized as having antibacterial properties.