The views that I express here may be controversial but I've always believed that the USA were way too soft on Iran during the hostage crisis. OTOH, I acknowledge that a ground invasion was impossible. So, what if the USA, probably under a different President than the ultra-pacifist Jimmy Carter, sent Khomeini an ultimatum, saying that either he ordered the release of the hostages in 48 hours or there would be war? This, IMO, is what should have been done.
Would Khomeini have accepted the ultimatum? A piece of evidence that he would is that after the 1980 election, Carter told him that, from 20 January 1981, he would be dealing with someone much tougher and right after Reagan was sworn in, he released the hostages.
If not and it came to war, I should note that the war that I'm refering to wouldn't be a ground invasion, it would be airstrikes against Iranian military, governmental and oil infrastructure and a naval blockade.
What would be the effects of the war? Unlike what many people think, I don't think that the hostages would be killed, because that would give the USA an incentive to destroy Iran as a state.