Well, Grant's father in law was a pro-slavery Democrat, maybe the father in law could influence him a bit, but I am pretty sure Grant was no fan of slavery even when he married Julia, even if he didn't express it as such. Hell, even in the 1850s Grant was in enough depressing and dire straits that he could possibly - POSSIBLY - find slavery attractive enterprise. But I still think its hard.
For Lee, well, he just needs to pick country over state.