As suggested in the earlier post, Julian's survival past 363 AD could have made a major difference; however, I do not see total elimination of Christianity or its reduction to fringe status at this late date. Not to mention that, should Julian have prevailed and had paganism as the official religion of the Empire even in the VIIth century, the unifying force that Christianity presented during Persian wars of Heraclius (who is sometimes considered as "the first crusader" due to his use of Christian rhetoric to unify his Empire against the Zoroastrian Persians) would not have been there; with a large and far-flung empire paganism, due to its relatively flexible nature, would also not be able to provide a centralized focal force for the population to rally around.
Another alternative is an earlier POD, possibly somewhere in the Ist or IInd centuries, where Christianity either never fully develops, or stays a regional religion... say there is no exile of Jews from Palestine after rebellions in Judea in 67-71 AD. As such, one potential outcome is that the fledgling Christian community stays relatively confined to Palestine and Syria, and does not spread out to the rest of the Empire in meaningful quantities as in OTL. In other words, Christianity would be a fringe cult, worshipped in parts of the Middle East, but not anywhere else. That would still allow for either Islam or something very similar to it to develop due to the strong presense of monotheists in the area, but could leave the rest of the Roman world relatively untouched.
Providing Byzantium is still pagan it is likely to be somewhat more tolerant of various religious minorities as long as they are willingly accepting Roman supremacy - with Christians being an insignificant minority as opposed to OTL (where much of the persecution happened due to the fact that the Christians refused to sacrifice to Fortune of Rome for the Empire's preservation, therefore the Emperors of the IIIrd century had frequently associated Christianity with dissention and a potential source of insurrection), and most pagans having no qualms about offering sacrifices for the well-being of the Empire, the Empire would be relatively cohesive, and better able to present a unified front against an enemy with a hostile attitude to anyone not believing in one god. Ironically enough, Christians and Jews would be more likely to side with the invaders, who, after all, are also monotheistic. I would imagine, however, that overall the Empire might do better in the short run, but in the long run the differences between different regions could lead to further fragmentation, especially once the Roman/Byzantine military power is reduced and/or collapsed.