The problem isn't necessarily the "undisputed" cases, though - it's that, especially as people conquer each other, people have sex. Sex makes babies, who are neither obviously one or the other. Now one way of socially classfying the "edge cases" is the infamous one drop rule, but history shows that this is not always what is done. Not to mention that legal categories of races tend to be hard to sustain long term in none hypodescent societies - as an ethnic Filipino of mixed descent, depending on the categorization scheme I could be as white as Mariano Rajoy or as Asian as Lee Kwan Yew.