Not to mention the fact that the actual number of citizens (which automatically excludes children, slaves, women, and foreign residents inside cities) of any individual polis was never huge. In Greece, Athens was for a time the most populous city, and the city itself probably never exceeded 35,000 inhabitants. Attika may have had about 200,000-300,000 total inhabitants, but that is including all of the non-citizen categories I just mentioned. When you read Thucydides' history of the Pelopponesian War, notice that Athenian armies are vary rarely more than a thousand Athenian men strong (though they often include detachments of allies as well).
Compare that to the city of Babylon, a centre for several different Empires, which already had 140,000 inhabitants by the 9th Century BC. Or Nineveh, which probably exceeded it at the height of Assyrian dominance, along with other huge Assyrian imperial cities like Kalhu, Nimrud etc.
Assyria was able to found an Empire because it was already the largest state in the Near East, because it had the largest and best equipped armies, and because it had a number of administrative technologies that enabled it to control populations under its dominance.
Athens, whilst from the 5th Century to the mid 4th Century the largest city in mainland Greece, was never so large in comparison to the rest of Greece that it was militarily dominant in this way. And the social technologies of Greece were dedicated to the control of the populations of single cities at a time under institutions and civic duties/responsibilities.
It's got nothing to do with racism, I've spent years studying Ancient History with a focus on Greece. It's got everything to do with the specific situation of Ancient Greece; its history, its culture, its political structure, its geography, its population distribution, its amount of arable land. It's not an insult to Ancient Greeks or Athenians, I'm not sure at what point you decided that if a group of people are not capable of creating an Empire they must be worse than other people.
If you want to point to Hellenic states that might have been capable of Imperial regimes, I'll point you at Macedon (who actually did establish one), Epirus (who nearly created one under Pyrrhus), Syrakuse (who at one point dominated the Greek cities of Sicily and had the largest population of any Greek city in the world at that time), maybe Taras (Largest and most militarily powerful Greek city in Italy after the fall of Sybaris). Why are these places different to Athens, and mainland Greece? Each of them existed in a completely different political environment, geographical environment, and had a completely different set of peoples to contend with.