Gingrich is just too...well....Gingrich. Just the mention of his name draws out some internal uncomfortably in the heart of men (well, moderate and liberal men). He's too polarizing.
Plus, he cheated on his wife with a women in the 90's who he later married. With that, you have hypocrisy (Newt went after Clinton, after all). You also have, independent of that hypocrisy, a smear on moral fiber which won't help with "family values".
So, you have a candidate who makes everybody not a Conservative rather uncomfortable, who has no moral standing as far as politics is concerned, who had gained low approval towards the end, who had spear headed an unpopular bid to remove Clinton, along with some some ethics scandals he garnered along the way, who is going fresh out of all that into a bid for the White House.