Having the U.S. government housed within a state is no big deal. The status would be the same as with U.S. military installations, immune to certain state regulations. There was a time when uniformed servicemen could buy beer on base, even if they were under the state drinking age.
I think it was a bigger deal for certain of the Framers in 1790 given what Madison said in Federalist no. 43 but it's not as big an issue now and maybe if a nice enough package is in the offing...
The biggest issue is Southerners not being happy with this frankly.
If the capital were to be located in Philly (or any other city in an existing state), I think that that is exactly what would happen, it would be in the state (and maybe the citizens living in the capital city would be considered citizens of the state and vote on those elections and be able to vote for the president etc), but the city itself would be considered a special zone and exempt to certain state issues as well.
And Roger II, I think the biggest issue WOULD be the southerners. Would they have been able to be okay with a "northern" capital from the get-go?
Now, if the capital at DC is abandoned after the War of 1812, Philadelphia is a good candidate and it wouldn't have been that long since the capital
left Philly anyway, and by that point the Pennsylvania state capital would already be gone.
If the capital starts out there from the get-go, there will likely be more thought into the city's development as the capital. If it moves there due to the War, I think that it would take a bit before a more uniform plan is put into place. I like the idea of a Haussmann-style renovation of the city taking place at the end of the 19th century. Could definitely see that sort of thing happening.