I don't have my sources nearby but as far as I remember, the conquest was a close run thing, with the Vietnamese accepting to pay nominal tribute afterwards, although it may be an atavic reaction to paying tribute to whoever comes from China.
My take is that the ongol could take it but certainly not pacify it, unless they get massive forces and coopt the elites. And even then, you have to coopt EVERYONE and then engage in cultural cleansing while still coopting the people.
Just look at the war of 1407, that's the closest China got to taking Vietnam back, and that was a ten year effort involving a 280.000 soldiers invasion force (those are the realistic figures). Otherwise, you get insurgencies popping up in Nghé An and in the mountains and smaller deltas. Vietnam was extremely agrarian and so very decentralised. You have to fight valley by valley to get it done and always run the risk of insurgencies popping up in your back.
The French only managed it via overwhelming force and pacifying an area torn apart by rebellions. And even then, couldn't keep it for long, despite elite cooptation through Protectorate.
However, if the Mongols were really clever, they'd make an alliance with Ciampa and the Lao to occupy territory, the Ciampas were still fearsome at that time I believe.