It would probably become impractical before 1900, but because of how hard it is to change the Confederate Constitution, it may not be actually outlawed until later on.
At least we'd have no Klu Klux Klan (There would be no Republicans to run out of the South) and maybe not the harsh Jim Crow...