Thanks for the help, everyone.
To clean the bottles, I usually run them in the dishwasher (no detergent) right before I'm getting ready to bottle. I open the dishwasher for the first time as I start bottling. Is this not good enough?
I'm assuming I do have a sanitation problem, if that's the common thread here.