I think <gasp> that Turtledove got one thing right in his South won the Civil War series: DC stays as the official capital, but all real work gets done elsewhere...perhaps inaugurations and a ceremonial opening of Congress, and a few offices. There are small "embassies" there with big "Consulates" in the real capital. Each government office has an office in DC also. If you go that route, Philadelphia isn't a bad choice for the main administration, as it's not too far away...
So basically, like the situation in West Germany, where West Berlin was still the "official" capital but the
real center of government was in Bonn?
That sounds doable.
Now the question is, where is the new capital going to be? Philadelphia and New York both have history behind them, but they're both on the East Coast. There might well be a need to "balance" the eastern and western states, like how the site of Washington, DC was chosen as a balance between north and south. At the same time, any location too close to the US-CS border is out of the question. In that case, I'd expect the center of government post-war to be along or near the Great Lakes. Chicago is an obvious choice, but the state of Illinois might be too associated with "the man who lost the war", and the city might be too far west for the East Coast to accept. I liked the suggestion of Toledo, Ohio; other choices might include Cleveland, Youngstown, and Erie, all of which are along or close to the Pennsylvania-Ohio border (a clear boundary between east and west if ever there was one).