A substantial number of white Southerners were deathly afraid of slave insurrections - in whch they might all be killed. As long as the US government stood behind the state governments, which stood behind the individual slaveholders, no slave insurrection could succeed, and so almost none were attempted. If that backing was withdrawn, white supremacy in black majority areas could evaporate.
A comparable situation would be the Communist governments of eastern Europe. Those regimes were detested by nearly all citizens of those countries. But everyone in those countries knew that overthrowing their Communist rulers would be answered by Soviet intervention with overwhelming military force. When Gorbachev showed that the USSR wasn't going to act - all of the Communist regimes evaporated.
What the more paranoid slaveowners feared was that Republicans would use the Federal government to infiltrate "abolition fiends" into the South to foment slave insurrections, while at the same time denying Federal support to any slave state government facing insurrection.
In the longer term, "Deep" Southerners (from South Carolina and Mississppi, where slaves outnumbered whites, and Florida, Georgia, Alabama, and Louisiana, where the slave population approached 50%) feared that if anti-slavery Northerners controlled the Federal government, they would inevitably destroy slavery, regardless of constitutional restrictions. That would leave white Southerners submerged in a free black population whose supposed natural tendencies to vice and corruption would run wild. Lower-class whites especially feared this, as they would lose any superiority of position. (Slaves were a minority in Texas, but the white settlers there were largely from the Deep South and shared its attitudes.)
Whites in the Upper South were generally much less fearful - of slave insurrection and of "nigger equality". There were exceptions: areas such as the Virginia Tidewater where many counties had slave majorities. The plantation owners in these areas were among the wealthiest and most influential men in those states, and pulled the states with them.
The South acted in 1860 because the extreme paranoids feared immediate disaster, and secession advocates generally thought it was the right moment. There was widespread fear of Lincoln, which would ebb after he took office and governed like the moderate ex-Whig he was. The ex-Whigs of the South were the mainstay of Unionism there, but as of 1860, they were a disorganized rabble, with no party or real leaders.
If Lincoln was allowed to take office and govern, he would use the considerable patronage power of the presidency to recruit Southern ex-Whigs to the Republicans, or at least to organized Unionism. This would build up a powerful anti-secession faction. Secession was a hard sell, even in 1860-61 (outside South Carolina). If the Republicans had time to consolidate power, it could become impossible.
www.archives.gov
