After four years of relatively unorganized (and frequently corrupt) government, the anti-Jacksonians (or Republicans as they came to be known- I happen to like the name, and it would point out some of the key political differences between the anti and pro Jackson forces) have organized themselves. They decide to support John Quincy Adams, with Henry Clay as his running mate. The '28 race is close, but Jackson's bad government allows the Republicans to win.
Andrew Jackson will be remembered as a mediocre president, whose worth was largely in helping to provide the rallying point for the anti-Jackson forces to create the Republican Party and its "American Program". The next 12 years (2 terms of J.Q. Adams, 1 for Clay- poor Clay, but maybe he comes back to win in '44) are spent improving the country.
Well, I'd certainly have voted for such a party. (And I agree on the name, for what it's worth).
However, this "American Program" would not be without its detractors. For one, it would rely on support of the Bank, which was
hated in the West and in parts of the rural East. If backwoods populism doesn't get expressed in Jacksoniansim it won't go away. And while the Bank was good in the sense of providing central controls, it did encourage an awful lot of nepotism and could cause potential corruption.
Furthermore, what happens to Texas? Sam Houston may not end up in Texas if can avoid political humiliation in the 1830s. If Jackson is out of power, then he well may because Houston is Jackson's best hope for a political successor. And Houston is essentially an Indian-liking Jackson. Texas is still going to revolt in 1835, unless something keeps Santa Anna from revoking the Constitution of 1824. Without Houston, though, the Texans may be defeated. If Texas doesn't win its independence, then the trajectory of Western expansion and thus of the slavery debate will be significantly altered.
If Nat Turner's Rebellion can be avoided, then Virginia may pursue some form of gradual manumission in the 1830s, particularly if John Floyd doesn't have competition from Jacksonian Democrats.
There's really a ton of stuff that can be changed.