You Don't Know, So Why Should I?

I'm starting to get really annoyed with people who get really annoyed with me for not being able to tell them where another store/restaurant/whatever is. I don't mind them asking, but I mind them getting mad at me for not knowing.