I think slavery would have taken a hit but would not be extinguished.
Plenty of other crops (tobacco, indigo, rice, sugar) could/would still be grown at a profit but not as great a profit. The value of slaves, therefore, would drop considerably. There would be less slave importation (both legal and illegal) from TJ's time to 1861 and that would affect populations. More would be voluntarily freed from 1800 to 1865 than otherwise, creating a larger class of free blacks, which would also affect the outlook of the institution.
As much of the anti-bellum south culture was based on the highly profitable crop, the overall outlook of the culture would change somewhat.
Perhaps some of the western slave states like Arkansas, Tennessee, Kentucky, Missouri, Texas, etc, would be less likely to implement slavery in the first place (or eventually emancipate) as cotton was the main large plantation-scale crop relevant to those areas (no tobacco, rice, indigo).
Also, tobacco is a crop even harsher on the land than cotton, if that is possible. It quickly wore out land. If tobacco isn't grown to the west, that would create great bouts of surplus slaves. Some could be bought cheap and used as labor on smaller scale but the overall prestige of the institution would change.
If the end result of these changes by 1861 was (I'm making these numbers up naturally), fewer blacks overall due to the effective premature end on importation of slaves, 1/2 to 2/3rds the number of enslaved blacks, a richer mix of free blacks, two or three fewer slave states, several more that only have nominal populations of slaves (Delaware, Kentucky or Missouri-style), this would have a marked affect on American history.