To not travel to another country because you don't speak the language is only robbing yourself of seeing the world. Nobody wants you to know their language but if you show a sincere effort to communicate in their language, even at a basic level will earn you a lot of patience from the locals. Also, to expect someone to know English just so they can visit the states is crazy. They are visiting, not trying to gain citizenship.
I say this from experience and living in 8 different countries and doing my best to use their language.
Also, there are places in France that welcome Americans with open arms and meet us more than halfway when it comes to language. The biggest reason for this, we are not German and they still remember what our grandfathers did back in WWII.