Byzantium went through cyclical stages: Glory, Stagnation, Disaster, Recovery, then Glory again. They had very good leaders during the recovery and Glory periods, but during the disaster periods they also had some of the worst leaders on the planet in charge. Overall, though, they weathered this rollercoaster of ups and downs, until eventually they simply faced too much at once (Having the Angeloi, Greedy Crusaders, Turkish invaders, and Norman raiders all at once just seems a little unfair). Byzantium had weathered similar storms in the past, but this time, disaster was followed by further disaster and when recovery came, it could no longer be followed by glory.... This was by no means inevitable, it's just how history turned out.