Well, the power and territory of the Empire has waxed and waned over time. At times it didn't look very imperial at all. (The most obvious being the time you mentioned in the OP when Rome was sacked by barbarians.) Yet like the phoenix, it always seems to rise again. All we need is to take one of the times when the Empire was particularly weak (the Western Crisis of the late 400s, the humiliating territorial losses to the Crimeans in the 984, the various rebellions in Hispana and North Africa in the late 11th Century, or even the more modern Pan-American Wars) and have someone stamp it out completely. People always try to point to the Persian invasion in 950 as a good point for this to happen, but I disagree. While the Persians had a lot of success in Asia Minor, they were way overstretched and the Empire was at one of its greatest extents in decades out west, so even if the unthinkable happened and Constantinople was captured, the Persians could never have held it.
Maybe if you could get a stronger Persia during the 1000s, they could have taken advantage of the rebellions and crushed the Empire. I mean the Romans weren't at their strongest even before the rebellions broke out. They'd lost most of northeastern Europe to the Crimean Empire and the Vikings had thrown Britannia into chaos. If Persia had been as strong then as they had been in 950, they could have easily swept through the Holy Land and into Asia Minor. The Romans would have had to let the rebels go in order to pull back their troops and defend Constantinople. Even then it might not have been successful given the sad state the legions were in under Gustavius II. Fortunately, the Persians were too busy with their own internal conflicts and couldn't take advantage. Not sure how to butterfly these conflicts, but if you can, it could conceivable end the Empire.