I was more thinking "Okay, so we arrived too late to reinforce it, but we can take it back." sort of thing. What stops besieging Vienna to get it back?
Well, they had troops to defend a siege and troops to relieve a besieged city. The requirements are much reduced from those necessary to take the city. The Ottomans brought the latter sort of army, and would not
need to defend the city. They could simply take the field facing the approaching armies, at which point those armies would withdraw or be destroyed.
It seems kind of odd that if the city is taken that the Hapsburgs will give up and not even try given its importance. Even if they're not overly confident in their forces relative to the Porte, would they be willing to give up this much just to avoid a possible defeat?
It's not a matter of cowardice, just practicality. If the city goes that's the war. Why didn't the Nazis just retake Berlin?
Vienna's not an easy spot to take, and the Hapsburgs had to struggle just to defend it. Not only would retaking it require a great deal more effort and strength, but because it represented the greatest single Hapsburg asset they are less able to muster even their previous strength. The dynasty lived off the political capital of being defenders of Europe in OTL - without the benefit of this they would have less support and eventually less territory.
Not arguing on this part, but could you explain how this can't (or at least would be improbable to the point that sustaining an argument that it could be would be optimistic at best) be done without Vienna?
Without Vienna you can't control the immediate regions just south, southeast, and east of the city. Essentially, it let Austria control the modern southern and eastern borders it has in the modern day, and a little beyond. This in turn gave Austria the opportunity to get involved in Slovakia - without the city the region would cease to be in play. Not that some trouble couldn't be stirred from Bohemia (Poland, not so much), but without Vienna they have no foothold on the plains. Essentially, the city determines whether the Turks have freedom of action within the old Kingdom of Hungary.
They failed to conquer Hungary's periphery not because the areas were particularly hard to take, but because they couldn't put a big army anywhere except in front of Vienna. Commit your army to Slovakia or Slavonia, and the Austrians can strike at your rear. Turn the army around to fight them and they'll just run back to Vienna, so you might as well just take the city in the first place. Which, after all, is what they tried to do. The remaining bits of Christian-ruled Hungary were unable to return the favor for Vienna at the same scale (hence the ability of the Ottomans to attempt Vienna at all). If the Turks hold the city, they're essentially the only state with substantial power projection on the plains. They can then digest the rest of Hungary at their leisure.