As Uff has said, Norway and Sweden had very differing national interests at the time.
Norway was oriented towards merchant shipping, fishing, whaling and a (rather small) industry based around that. Norway can be invaded from Sweden, but it is difficult. In many regards, Norway is an Island. And as such, they needed only friendly relations with Britain, and no-one could invade them. Especially during this era when Britain ruled supremely. Norway has no natural or arch-enemies during this era.
Since the Black Death, there has been no local Norwegian nobility and no landed elite. Sure, there were Norwegians that were knighted in the service of the Danish King, but there were no strata of big, conservative landowners in Norway. The Norwegian elite were intellectuals and merchant magnates and they were almost to a man liberal. They saw Britain and British policies of free trade and getting rich off of being a merchant as the pinnacle of civilization. They had no need for expensive armed forces or even a large navy.
Thus, the Norwegian elite were liberal, free trade, merchant-oriented, British-friendly, anti-military (or at least wanting a very cheap defence so they could pay less taxes, since they did not percieve a need for a strong defence) and with little regard for heavy industry.
Scandinavism never really took root in Norway - it was a Swedo-Danish thing with Norway tacked on because their foreign policy was controlled from Stockholm.
As a comparison, Sweden was industrialising along the traditional iron ore trade, opening new mines and starting a metal industry of its own. Sweden had a large land-owning and then industrial elite (which sometimes, but rarely, overlapped) and was generally conservative for a long time. The Swedish elite wanted protectionism, was conservative, military friendly (and even inclined to foreign adventures - Sweden did send troops to Denmark 1848 and planned to do so 1864 and did want to partake in the Crimean War, but only as part of a larger coalition). Sweden had Russia as an arch-enemy - a country with a strong fleet (3rd largest at times during the era) and a land border. Sweden was friendly to Prussia (later Germany) and had extensive economical, academic and military ties to the country, with little such to Britain.
Even without nationalism thrown into the mix, the interests and priorities of Norway and Sweden during the era was completely different.
To make the Union last, you need to make the Norwegians Think that they are better off with it than without it. They need an external threat only Sweden can help them against, they need to have more similar economic interests as the Swedish ones, they need to think less of Britain as the only friend they need (a weaker, more hostile or more erratic Britain will do it).
In my "A different Finnish War" I have Sweden be more in Britain's camp post-war, with better relations and a longer Alliance. The Norwegian elite is carted off to Sweden to be part of a central government rather than be allowed to create their own parliament. They are also more economically successful due to being able to get through a hole in the Continental Blockade. This allows them considerable influence on Sweden and gives them the opportunity to make Sweden a bit more liberal.
There will be tensions, but in this case, I think Norway can stay with Sweden.