And also, it's doubtful that all the Khoisan would be killed or enslaved, but they weren't considered a "middle class" at all. Their conflict with the Dutch was kind of a lopsided version of cowboys and Indians, except that the Khoi almost never won. The Dutch didn't consider them human. That said, it isn't likely that all of South Africa's Khoisan would die en masse, but you probably would see a situation very similar to OTL, with only the Cape Khoi being exterminated and the rest driven into mountain ranges or deserts.
I must explain a bit more. I remember reading that some of the survivors bred with the White settlers in South Africa and their descendants were allowed to hold office(Though I'm probably mixing things up). Not the Khoi themselves.