They are rejecting you. They might not know all about you, but they're rejecting as much as they do know. Maybe the way you drank your coffee or the way you talk or the way you walk. Whatever it is they're rejecting it.
I sympathize with the idea that the best response is to brush it off. On to the next, it makes no sense to get invested in someone you barely know. But I think it's harder to do it when you have no history of success and all you've really known is rejection.
For this reason, I really think it's best for people to get some dating experience in their high school or college years. Those are the years where you really can get to know people without awkward dates with people you met at the grocery store or a dating site.
This is kind of a rambling post, brought on by some recent events in my life. So it's not my usual post. I'll try to write something useful next time.