The bureaucratic machine of the Imperium was horrible, overly complicated and corrupt and the Inperators had almost no idea of the state of the Imperium thanks to it.
Huh? The Roman Empire of the first and the second centuries had serious structural problems, most glaringly the lack of bureaucracy that were papered over by weak external enemies, a fairly prosperous economy, and a few strong personalities that became emperor. The moment these things changed-weak or crazy emperors, strong enemies on the Rhine, Danube, and Persian frontiers, and economic decline, the structural weaknesses and internal contradictions of the systems were laid bare and the empire imploded and very nearly collapsed in the third century. The reason any rudimentary form of bureaucracy even began to come into existence was because it was completely necessary. The renewed pressure on the Persian frontiers and the the expansion of the army under the Severans required new revenue streams, especially at a time of economic and population trouble. The old system, with provinces managed entirely by a senatorial legate and his senatorial quaestor were insufficient for this task. Even at the height of the empire in the second century, the senatorial aristocracy only barely had the numbers to fulfill the duties of government required of them. To operate effectively, the Senate always had to have a significant number of their body in Rome at all times to fulfill their legislative and judicial duties, while these senators were simultaneously required to churn out enough former consuls and praetors to run the affairs of government in the provinces. Every province needed a senatorial governor (and in the case of senatorial provinces, an imperial appointed legate, also coming from the senatorial class, as well), as well as senatorial legates to command each legion (lest any individual governor have too much power).In addition to this, there were senatorial urban praefects, curators, a juridici in some provinces to manage judicial duties (such as in Tarroconensis and Cappadocia), and of course there were junior senators working as assistants (adiutores) to senior magistrates.. By the time of Marcus Aurelius, 160 senatorial officials were required to manage the empire, and a significant number of them had to be of consular or praetorian rank. This was barely sustainable in the best of times, and was completely unsustainable once the empire faced significant internal and external pressures. For example, by the second century there was often only one senatorial official per 350,000-400,000 inhabitants.
This is where the equites came in. As the bureaucracy needed to be expanded to manage the empire and the challenges it faced, it was more efficient to appoint equestrians to these roles. This was particularly true once citizenship was expanded to everyone. These posts required long tenures, developed a hierararchy and cursus honororum, and provided a form of efficiency and professionalism that was not present in the early empire and could not be with senators. It became a quasi-patronage based quasi meritocratic system rather than the pieced together ad-hoc bureaucracy of the early empire. Equites rose through the administration based on local patronage, to some extent on ability, compared to senators, who's careers relied on personal patronage of the emperor. This was already the de facto situation before Diocletian and Constanitne systematized this, and also led to the de facto division of civilian and military offices, as each developed their own cursus honorum. It created a semi-professional civilian bureaucracy alongside a semi-professional military hierarchy that was lacking in the early empire and was necessary in the later empire, especially since the empire could not realistically be micromanaged by the emperor himself as it had been in the early empire.
As for the Dominate in particular, I don't see where the clash between civilian and military officials really comes into play. At the most basic imperial administrative level, the civilian dioceses became the strong foundation of imperial government. The diocese effectively were the ones managing the government on a day to day level. The vicarius and his staff oversaw tax collection and managed judicial functions in the imperial stead. They were better able to dole out justice and manage the complex legal jurisdictions, which often overlapped. In effect, the vicarius was able to provide surveillance over local officials. This local bureaucracy functioned efficiently no matter who the emperor was or what the imperial politics at the top were, and provided revenue and legal and administrative continuity regardless of the political situation and the frequent changing of hands of dioceses between emperors. This was immensely important to the stability of the late empire.
As for the praetorian prefects I can't think of any situation where they played any significant political role after Constantine-their job was to manage the military's finances, and maintain the infrastructure and postal system. They all required any number of lesser officials to carry out these activities. Again, this systematized and proffesionalized what had been an ad-hoc system and allowed for the empire to function relatively smoothly even when there was significant turmoil at the top.
What you are referring to with regards to tension between civilian and military officials really isn't that. It's simply tension and squabbling within the emperor's comitatus, and in particular the most prominent posts of said staff. The financial bureaus created by Diocletian and Constantine (such as the res privata and the sacrae largitoiones) weren't controlled by anyone in this separated civilian and military hierarchy, but were always controlled by members of the imperial comitatus, who owed their position and loyalty to the emperor himself. The magister officiorum, and the magistri militi were both a part of the emperor's comitatus. I fail to see how this would change without the bureaucracy that formed in the 2nd-4th centuries. The senior military and civilian posts were always going to be loyal men of the emperor, chosen by him, and they were always going to squabble among themselves and potentially engage in civil war if the opportunity presented itself, just as was the case at any point in the Roman emperor. The only difference now was it was not the governors or commanders of the individual legions that were revolting or seizing power, but the people with whom were now invested with real military and political power.
ulianus had to purge it when he became Augustus and his clashes with the Praefectus were "resolved" by Constantius bypassing him and indeed it didn't impeded him for very long because it ended with him rebelling just after 4 years.
Julian had to replace it with his loyalists, mostly because the leadership in the eastern half of the empire never really liked him very much and represented a political threat. Which would be the case at any point in Roman history. As far as his clashes with Florentius are concerned, they are horribly overblown. Florentius devised a scheme to make up for a revenue shortfall, which was opposed by Julian and appealed to Constantius, who proceeded to take Julian's side. Regardless, any conflict between Florentius and Julian is not indicative of conflict between the civilian and military leadership, but is the result of the fact that Constantius appointed his own men to high level positions in Gaul surrounding Julian, specifically to constrain him and make sure real authority lied with his own trusted officials.
I'm rather curious about what good are you referring to, under Diocletian you had the division to avoid a General having too much power and resources and yet it never stopped civil war and the military officers usually ended in control of both civil and military power in their areas so the "division" was nominal at best.
That's not why a dedicated civilian bureaucracy separate from the military was created, though. As I elaborated above, it was created because it was necessary to manage the empire, and was above all much more efficient than the ad-hoc system that preceded it.
Also, where's
@Basileus Giorgios when you need him, he's far more knowledgeable about the dominate than I am.