Slavery and Civil Rights in the US after Confederate victory

Something which seems to be frequently glossed over in discussions and timelines is the fate of slavery in the US after Confederate victory. Even if the South succeeds in seceding, there is still the matter of the border states. Here are some stats (from 1860) to elucidate the situation.


Slaves as percent of population: 1.6%

Slave owners: 587


Slaves as percent of population: 19.5%

Slave owners: 38,645


Slaves as percent of population: 12.7%

Slave owners: 13,783


Slaves as percent of population: 9.7%

Slave owners: 24,320

In Delaware, slavery was marginal, and I think it is quite likely that some sort of compensated emancipation scheme could be worked out soon after the war. But it wasn't as marginal in the other states. In OTL, Maryland did pass a new constitution abolishing slavery in 1864. But that vote was very narrow and could easily have gone the other way. Given that slave owners arguably held disproportionate influence over state governments, how long could slavery last in those states? I think it's unlikely the US would hand over the 'contraband' that had fled north, or the former members of the USCT, and the southerners might be reluctant in demanding such a return. But there still will be slaves who try to flee north. The Fugitive Slave Law is dead, but would any Fugitive Slave Treaty replace it? Abolitionists would certainly oppose it, but many poorer whites might support it, seeing escaped slaves as competition that would drive down wages.

Eventually slavery will end, but the former slaves wouldn't have any rights. Few of the free states in OTL offered black people any rights, and with no reconstruction amendments, that won't change. Since there will likely be a backlash against Lincoln and the "Black" Republicans, the situation might be significantly worse than in OTL. In the border states, I would not be surprised if something like OTL's Black Codes would be implemented after emancipation.

Might there be more support for colonizing former slaves elsewhere? Outside of the border states, there were relatively few black people, and the northern economy did not depend on black labor the way the southern did. And, at least in the years immediately following the war, many northern whites might blame blacks for causing the war in the first place. So there might be strong public support for colonization.
I'd assume most(border states with slavery) would join the Confederacy, however that depends on how successful the war is.
Kentucky and Missouri are the most likely to join the Confederate States given their political, cultural and economic ties to the rest of the South and the fact the Confederates had their own shadow governments and had substantial presence there. Of course, Kentucky and Missouri as part of the Confederate States depends on the POD involved. That leaves Maryland and Delaware as the only slave border states to stay with the United States. Maryland would have a sizable black population especially concentrated in it's largest city Baltimore. Delaware would have it's own black population but smaller than Maryland.

For blacks and potential black migrants living in the United States, they would have next to no rights whatsoever given the social attitudes of most whites at the time.