New England was almost ready to split before the War and I don't think it is pushing it much to envisage a scenario where everything north of New York MAY go British (or at least Canadian).
New England was entertaining the idea of seceding from the US, but that in no way suggests it would promptly, or ever, join up with Britain or Canada, merely that it would secede from the US. If it did so, and Britain was able to extract terms from the US, none of those terms would include annexing the newly independent "New England" but rather forcing the US to rescind their claims to those lands and accept a British Guarantee of their independence.
New England will be heavily influenced by Britain, of that I have no doubt, but it isn't going to be annexed by Britain any time soon. And after all, why would Britain want to annex them? Whatever paltry sum Britain would gain in taxes or manpower is vastly outweighed by Trade.
But even this isn't going to make the US fall apart. If anything, this will probably strengthen the bond between the remaining states, because now they have proof positive of the idea that if they don't stand together, they'll just become pawns of Britain once again.
What it will do is nip the Slavery Debate in the bud, since now there will be an overwhelming majority of Slave States...