I feel like this entire question is predicated on a faulty assumption: that Europe, Japan, or China were ethnically homogeneous. Now, some other people have made points about multiethnic empires like Austria-Hungary, or discussed the multinational character of the United Kingdom or Switzerland (incidentally, Switzerland is a terrible example because its extreme federalism and mostly-homogeneous cantons makes it questionable how "multinational" it has been anyway)
Anyway, Japan, arguably, has been quite homogeneous for quite a long time, being as it was a fairly centralized island country with isolationist tendencies. That said, it can be argued that Japan was effectively homogeneous at the time that it "became successful", so we'll leave that aside - being ethnically homogeneous, it seems, does not prevent success.
Europe and China, though...even not considering Europe as a whole, individual European countries were often more diverse than modern people think, looking back at the homogeneous nation-states of modern Europe, and forgetting that they were achieved by massive migration, ethnic cleansing, genocide, and assimilation in the past couple of centuries. Bretons in 1700, for example, quite definitely considered themselves to be culturally distinct from Parisians, as did most folks from Languedoc, Savoy, Calais and Picardie...it was only a massive campaign to standardize the French language and culture under the Republic that saw the "homogenization of France". Fun aside: the French, and educated English speakers, like to mock a lot of English "mispronunciations" of French words. Especially in legal and technical terminology, this is not a mispronunciation, but rather reflects the pronunciation is the now dead Norman dialect. Germany was very diverse, including Germans of various stripes, various Slavs (e.g. Czechs, Poles, Silesians), Jews, and Baltic Prussians (at some point). Russia featured Slavs of many different groups, some of whom have been considered to be the same and some not (Russians, Ukrainians, Belorussians, Poles, Ruthenians), Balts (Lithuanians, Latvians), Finno-Estonians (Finns, Estonians

), Baltic and Volga Germans (very distinct despite both being Germans), and Mennonites too, Tatars, Georgians, Armenians, Chechnians...though maybe Russia, as the "sick man of Europe", isn't the best example. Sweden to this day has a lot of Finns and Finland a lot of Swedes, and Sami in both; the UK had the English and the Scottish and the Irish and the North Irish and the Welsh and the Cornish and the Manx and the Orknians, and even English and Scottish weren't very homogeneous, with Highlanders and Lowlanders and then Northmen and East Anglians...
But mass media and centralization led to more and more "standardization of nationalities", causing e.g. all Englishmen to become mostly the same kind of Englishman, all Germans to be the same kind of German, etc (taking some liberties, of course), and then the aftermaths of WWI and WWII led to a lot of migration, voluntary and otherwise, that really cleaned up the borders. But one should not assume a great deal of homogeneity in historical Europe; even small countries like the Netherlands saw a lot of feeling of internal division between Frisians and Hollanders and Zeelanders...
And China, too, was very diverse. Nowadays we like to call China a "mostly Han country", but even modern China has almost 10% ethnic minorities, and a lot of the dominance of the Han is due to assimilation and a gradual expansion of what "Han" means. Han from the Southwest and the Northeast of China, for example, look physically different (for the most part), dress differently (prior to the modern dominance of Western dress), eat different foods, and speak languages that are mutually unintelligible (though the same language in writing, in theory). The Han as a group draw their heritage from the Yellow River, but it's very clear from historical and genetic studies that China is not entirely peopled with Yellow River-dwellers who expanded into the rest of China, expanding into a vacuum or killing who was there and replacing them. Rather, Min Chinese have clear connections with Viet and other Southeast Asian peoples, northeastern Chinese ("Manchurians", not to be confused with the Manchus) have a lot to do with Mongolians and other steppe peoples, etc. I've also read a theory that the term "Han" began being used in the period after the collapse of the Han dynasty by all agricultural and urbanized people in China to contrast with the waves of nomadic herdsmen pouring in from the north, sort of like a word for "civilized" (or, for a better parallel with European history, consider all the groups calling themselves "Roman" - if Germany, Russia, Greece, and Anatolia had eventually been unified politically, and a common written language forced on their ruling and scholar classes, and then called the result Rome and themselves Roman, that would be about the same). Again, centralized government and mass media helped unify these identities (and the Cultural Revolution, of course - Taiwan, which escaped the latter, is affected by a very severe founder effect as most of the Chinese Republicans to migrate to Taiwan came from a very similar geographic and class background).
Now, it's hard to compare these diversities with colonial and post-colonial Africa. In some places, the terrain is very nonconducive to the kinds of large states or trade networks that would tend to slowly knit various groups together, and in general, I think very few of the pre-colonial states ever became centralized enough to begin a deliberate process of cultural standardization (as happened in Europe, China, and other places), and certainly none of them really had access to mass media - and, if anything, the colonial powers were usually more concerned about introducing new divisions than unifying old ones, in order to keep the people weak and disorganized.
So, to answer the question: the question makes an incorrect assumption; in fact, a better question might be, "Do successful states lead to ethnic homogeneity?", as such states are better able to use education and mass media to standardize culture, and to control migration to solve the rest of the "problem".
EDIT: Little Red Bean and Tripledot make my points, far more concisely, in the two posts immediately above mine, posted while I was writing my essay
