The biggest wrench into slavery is going to be the Industrial Revolution. Not sure how slavery would evolve if industry takes off in the South. One possibility is that slaves are made to work in factories, with plantation owners cashing out and selling slaves to industrialists, though the price of slaves will drop dramatically due to the dangerous conditions factories used. This could lead to any number of outcomes, not the least of which is slave revolts. When factory work is no longer seen as an opportunity but as a death sentence, people will revolt, and major reforms will be necessary or the CSA will suffer economic collapse and be forced to become an insular, somewhat agrarian society. This still would have it on the brink of collapse, with slaves constantly up in arms.
Freed slaves may carve out a state from the CSA of their own, but the damage to industry will be done, and reforms will be demanded all over North America. Sure, there will be a fight, but it will be with increased fervor and with more participants who don't want to be viewed as like the backward, slave-driving Southerners. Organized labor will have increased power, and perhaps either the USA or CSA will have a full-on Labor party or party faction.