I don't see any reason why new regime would change its capital. Only example for change capital after revolution is Russia but probably even that was ratherly strategic reasons than breaking ties to old system. France too didn't transfer capital away from Paris after Great Revolution. Perhaps they just re-name that. And Chicago is bit odd pick for me. Richmond or New York probably would fit better. And Washington had already its old administrative structures so it is bit pointless move away when them should re-build them again to new one.Part of me think that a victorious CSA would not be using any government buildings of the former United States, and instead, move their capital over to Chicago, to give themselves a clear break from the old regime. While transforming Washington DC's (Which they will most likely rename into something else) government buildings such as the White House and the Capital into museums, sort of like in the amazing 'People's Socialist Atlas' TL:
... Assuming the CSA, especially if they go Totalist/Vanguardist, don't go full 'Year Zero' on the place, denouncing the former United States as a bourgeois republic founded by racist rich white landowners using slavery on stolen native lands, hence, irredeemably evil and needs to be erased from history, as all of their monuments and symbols are inherently counterrevolutionary and oppressive against the working class.
Ending with CSA militiaman dynamiting the Capital Building while burning the Declaration of Independence and the original copy of the US Constitution in a highly choreographed ceremony in front of a film crew invited over from the Commune of France, complete with a children's choir singing the Internationale, the same way that the COF will do to the Brandenburg Gate if they capture Berlin during the Second WK.