I'm actually really curious about this now -- is there a way to actually settle the slavery expansion dispute early, in a way more permanent than the 1850 Compromise, so that the Know-Nothings' agenda can become the main talking points of American politics?
I don't think you need to settle the slavery dispute in order to make the Know-Nothings more electable. You just need to make the Democrats more dominated by southern slaveholders to the point that the Northern Democrats split (as happened in 1860 in OTL) and join the Know-Nothings: the Know-Nothings could paint themselves as a moderate force that says "We're not all about slavery" in between the anti-slavery Republicans and pro-slavery Democrats, and can compete with the Republicans in the North and the Democrats in the South, while those two parties only seriously compete against each other anymore in a handful of states. This would take a few electoral cycles and the slavery question to be pushed onto the back burner though.
To see what I mean about Fillmore doing much better in the 1856 election than people tend to think, and effectively competing with the Democrats, see
here and
here. Who knows how well they could do with a candidate more charismatic and interesting than Fillmore? (Though name recognition may have helped OTL).
OTL I believe the Constitutional Union ticket in 1860 incorporated the remnants of the Whig and Know-Nothing parties, which gives you an idea of their 'shut up about slavery either way' attitude.