I as an american have had exactly the same problems. I can understand, and respond to, the worst possible English spoken to me. But when i give a well-spoken phrase in Thai, Chinese, Malay, i get a crooked head stare. If I say it again, maybe 3 times, the same way, they get it. What happened? Meantime this has caused me to give up on Asian languages.
I was with a Thai friend in Thailand, sitting in a restaurant. An american woman comes in by herself and orders several items, speaking Thai. My friend said, wow she speaks perfect Thai! But the clerk barely had a clue what she was talking about.
I think its not pronunciation as much as it is psychology. You dont expect a certain person to speak the language, you freak out, you tune it out. Thats why someone is not being understood. Yes, tones make all the difference but people can figure at least from the context of the conversation whats MEANT but that doesnt seem to make any difference.