This is unrelated, perhaps off-topic, but when the Roman Empire was at its height, did most Romans in the West speak Latin, and most Romans in the East speak Greek, and they were wiped out by the barbarians et cetera?
That comes close. Of course, there were regional languages galore, but the general tendency was that those would lose importance not only during the height of the Imperium, but for as long as it lasted.
It very much depended on the degree of Romanization resp. Hellenization. And, on the other hand, it surely depended on the degree to which civilization had been smashed to pieces during the decline of the empire(s).
Basically the complete Western Roman Empire still speaks languages which are directly derived from spoken Latin ("Vulgar Latin"): Italian, Spanish, Catalan, Portuguese, French, the small Alpine languages - and the odditiy - Romanian (despite the province of Dacia only existing for about 160 years!). It is safe to assume, that there were regional dialects as it is usual in languages which stretch over such an empire. Just have a look at the many variations of the English language. Without the context of the empire, these dialects began to develop further apart. It is said that e.g. in France, the transformation into French had reached the point where colloquial Latin was not understood any more had been reached at some point during the 8th century.
Few languages reflect the languages of those people the Romans conquered: Basque, Cymraeg (in Wales).
Despite the extent of the Germanic kingdoms from the 5th century onwards, Germanic languages could only become important at the periphery of the Western Roman Empire, in Germany and Britannia.
The situation with the Arab language is different (likewise Turkish)- these conquerors were able to introduce their own language on the population of the Orient, Northern Africa and, for a time, Spain. I speculate that the sacral status of the Arab language helped this development.
On the other hand, also the Slav invaders on the Balkans were able to make their languages dominant in the former provinces of Moesia, Dalmatia and Illyria.