I suspect Roman settlers in Britannia is exactly why the possibility of retreating and trying to rely on an autonomous client regime was not considered. I am unaware of the Romans ever abandoning territory that had been long claimed by Rome--instances of retreat do come to mind--Mesopotamia of course, also I have the impression "Arabia Felix" (part of Yemen) might have been claimed as a province briefly, and I gather there was a stretch of territory beyond the Rhine held briefly then abandoned, and of course in Britain itself there is the territory between Hadrian's Wall and Antoine's Wall that, despite the fact that the latter Wall was shorter, seems to have been unsustainable--which suggests that conquering Picts was very unprofitable.
It did seem to me the obvious answer to a high cost of maintaining legions in Britannia would be expansion, to secure Caledonia and perhaps also Hibernia though the latter is tricky because it involves forcing a landing somewhere--I'd think perhaps politics with suitably interested Irish petty kings might give them a lodging to expand out of, and while I would expect Hibernia to be poorer in terms of extractable revenue than Britannia, still it is better land and more population than Caledonia! The concept would be that while expansion costs even more legions initially, once there is definitively no more borderland (the Norse would prove this concept erroneous eventually, but the Viking era is half a thousand years and more out, and they probably can't do a "Proto-Viking" era earlier) then the demand for legions and auxiliaries would drop, freeing them up for use on the Continent.
Apparently though, given the abandonment of the brief northward extension of Britannia, demand would not drop to zero! I guess that even at the peak of Roman power in Britannia, it was a matter of some towns and regions strongly Romanized, with a scattering of Romanized people elsewhere, but lots of petty power centers of Britons who did not revolt en masse, but were only grudgingly reconciled to the regime and stood to create some unrest unless cowed by the ongoing legionary presence. Also a certain amount of disorder in even more retail form, of bandits and pirates etc, really a smoldering form of the above Britannic resistance. If the Picts north of Hadrian's Wall were even less subdued, it might have seemed that the day of peace would never come, and that might or might not have been correct. My reasoning was that if conquering Caledonia and Hibernia were marginal, still there would be more revenue if scanty, and the net cost of peacekeeping force might come down after a few generations. But even if one could prove to Hadrian or whomever that this would happen in 50 or 60 years, it is the current Emperor's problem to pay for what is necessary today.
But meanwhile, unlike these other brief extensions of Imperial power, Britannia had been settled for some time by some Romans. I suppose few were attracted to the colder and wetter climate versus the Mediterranean or even Gaul, but some were, and some Britons Romanized. Once a substantial population of people who identified as Roman citizens were settled in Britain, the Emperor could not simply walk away from them or tell them to fend for themselves. Telling a patrician family with a history of service to the Empire that defending their Romanized lifestyle was now their problem would either result in their being effectively destroyed as Romanized persons if not existentially, or else, if these patricians and Romanized Briton gentry were to pull together and organize a sustainable Romanized state, then that's an example for all other provincial notables that they too can probably shrug off Imperial pretenses and run their local show their own way without reference to distant Rome. Either way, it is a huge blow to the soft power of Roman prestige and thus unthinkable to any Emperor as a matter of deliberate policy; even the eventual withdrawal of the last legion was not intended to be a permanent abandonment but a temporary expedient for some scheming general seeking the Imperial purple, reasoning that of course the legions would come back to Britain once the Empire was in his more capable hands. It turned permanent of course because Roman power was on the decline.
The point being, Rome abandoning or losing control of long held territory is in fact Roman decline.