The big question is what do you replace Islam with. The speed with which Islam spread suggests there were religious vacuums that were waiting to be filled and if that is the case then something else is going to come along and fill that vacuum.
The Arabs conquered very rapidly, yes, but Islam's spread happened much more slowly. Remember that Egypt and Syria probably kept their Christian majorities up until the tenth or eleventh centuries.
It seems to me at some point the Empire will collapse by a combination of internal or external pressure.
Why?
I mean, a collapse certainly
could happen, but there's absolutely no reason to assume it's going to based on trends ongoing in the sixth and seventh centuries. The late antique period saw, in all aspects of the empire (apart from the religious one, to some extent) a convergence and coming together that had never been seen before.
The Romans certainly
could collapse at some point in this no-Muhammad TL, there's fifteen centuries for them to do it in, after all. But I don't see it as being somehow predestined. As with China and Iran, there's nothing really to stop Roman civilisation from going on and on.
As for likely changes: without the shock of Arab conquests, there won't be any immediate pressure for the reforms of the military that took place between about 650 and 750, so we keep the Diocletian-style garrison and field armies in use. I reckon it's quite possible we could see more Exarchs placed in troublesome "semi-detached" areas, Armenia seems the most likely candidate for an Exarch.
The religious question could go either way, really: the only certain thing is that the Emperors are going to find it impossible to satisfy everyone. I think it's more likely effort will be put in to satisfying Egypt and Syria than Italy. We probably see more Justinian-style depositions of uppity Popes, with an attempt to keep the Papal office very firmly under the thumb of the Emperor/Exarch, although distance will always make this problematic. I can see local churches going so far as to sponsor imperial pretenders who have the right doctrinal views.
Sooner or later, accommodation of a sort will probably be reached. I don't see an Empire built by and for landowners collapsing internally, because this would go very much against the interests of these same landowners, who of course all operated within the same cultural and economic sphere, regardless of their views on religion. Any religious accommodation will be hammered out by local potentates in mind, although I think it'll probably be the ninth or tenth century before any sort of pragmatism wins out. With a POD after 600, it's going to be pretty difficult, I would suggest, for anything but a broadly Chalcedonian formula to end up on top, but who knows.
As an aside, where has the unusually incorrect term of "Rhomaion" come from?