When I was young, and occasionally still now, I often answered unwelcome sexual advances with something like "thanks, I am flattered, but sorry, you are not really my type". Their answer was, more often than not, "why not?" or "what is your type, why not me?". This did not sound like a rhetorical question, this sounded like a question for which they were expecting an answer.
Being somewhat literal-minded, I usually answered the question with a detailed list of the differences between the man in question and the man that I would find to be a desirable sexual partner. This always upset them and sometimes pissed them off.
I am not surprised that people don't enjoy hearing such lists. The question is, why were they asking? And what were they expecting to hear as an answer?