I understand, and I can see the union taking far south Florida, the most pro union WV counties under union control, keeping Kentucky and Missouri etc. Like Arnhem, however, Tampa and all points south, may be "a city too far".
That might be true. Since it's the only actual town between Duval county and Key West, and was traditionally ignored by North Florida at best.
If the Union holds the fort, and maintains the stations on both Egmont Key and Mullet Key, (which the local Militia, or what's left of it anyway, cannot contest, and likely would have a hard enough time trying to re-take Fort Brooke anyway) and we're talking a negotiated settlement, the Union probably could walk away with it without any real issues save a moderately disgruntled population.
EDIT: And even then, while there was no love lost for Tallahassee at the time in Tampa and Hillsborough, most residents really just thought the Federal Government utterly irrelevant to their lives, which it honestly was. Most of the actual opposition to the union in the area during the war had more to do with more pressing local disputes, like Union troops stealing livestock, or otherwise making a nuisance of themselves.