Take the sentence "We are going for a walk now”. If someone said that to me I would have no problem understanding it. Now imagine "We are going for a walk now” as a musical phrase. My aural skills are much more rudimentary when it comes to music, so chances are that I wouldn’t have completely understood the meaning of that musical sentence and would struggle with playing a similar phrase that conveys the same meaning.
Even if I was able to figure out how to play something close to "We are going for a walk now”, I would not be able to control whether I played it with a Southern accent or an Irish accent, or no accent at all. I’m not even sure I’d even be able to tell if the person who said it to me was Irish or American. I also may not notice a change in emphasis like "We are going for a walk? Now???". And I definitely wouldn't be able to automatically add a variation like "You and I will be taking a stroll at this time".
But at least I'm starting to recognize that playing by ear is more than just regurgitating the sentence "We are going for a walk now", and more than even knowing the meaning of that sentence. It's also about guessing where a person might be from based on his accent, being able to tell whether he is happy or angry, and knowing whether he is asking a question or making a statement (and probably much more than that too). The same listening skills that allow me to pick up on all this when someone is speaking gives me hope that I can also apply them to music.