It's because, in order to meaningfully apply OTL history (e.g. the Roman Empire) to such a fundamental change, we have to assume near-impossible levels of coincidence. Going by the infinite-multiverse theory, there will be some world where women are taller than men and people speaking Latin rule all of the Mediterranean basin, but such worlds are vanishingly unlikely compared with worlds where Liu Bang dies in battle and Latins rule the Med.
EDIT: And, since "vanishingly unlikely" is the defintion of ASB, any thread that attempts to combine fundamental changes with superficial constants goes there.