Being in the minority

In Hong Kong and Japan I realized that being in a place (a cafe, a subway car, etc) where everyone else is Asian and I am the only white does not bother me in any way. I stop even noticing it very fast. In contrast, being in places where I am the only white and everyone else is black makes me quite self-conscious and uncomfortable. I wonder why is that. The crime rates would be the obvious answer, but certainly not a complete one: I would feel uncomfortable if I walked into a place where a conference of, say, black ophtalmologists were held, and I am quite sure that ophtalmologists of any color are unlikely to contain a significant criminal element.