U.S. Presidential election 1864- Lincoln challenged by pro-war candidate?

What if 1864 was not between Lincoln and Little Mac, but between him and a pro-war candidate? What could lead circumstances to create such a split?

Checking Wikipedia, I guess there already was such a faction, the Radical Democracy party of Radical Republicans who nominated Frémont, but who declined to run.
 
Top