This may be ignorance/prejudice on my part, but I think the North African states would make a better job of subjugating a lot of African tribes then those tribes would of forming coherent nation-states.
Asante and Ethiopia prove you wrong, among a few others. It wasn't all ethnic tribes below the Sahara.
Also, maybe the North Africans head back to Europe? After all the Sahara is pretty much impassable, and if you go around it you end up with lots of hostile tribes, horrible humidity and horrendous disease. Europe has a wonderful climate, ample resources, and it's surviving population is reduced to knocking sticks together and living in fear that the Angel of Death is coming. Sounds good to me.
More likely, I agree.