The thing that also has to be remembered here is that if the US commits to a war with Britain they are taking men off the front line to 1) man the defences of the harbours and along the coast and 2) sit or attack on the frontier in Canada. Neither of those options is incredibly palatable for the US as it means they are taking men from other fronts and fighting a war on the other side of the country. This helps the CSA in the short and long term. Meanwhile when the crisis breaks out the US has to wait until spring 1862 before they can effectively organize a new army or attack Canada (so realistically they have to attack sometime in late June or early July) which gives the British time to prepare themselves somewhat better. No invasion of Canada is going to be a cake walk either, crossing the river is tough and the overland route through Maine and the Maritimes is a logistical nightmare. And considering the US needs some 100,000 troops (probably) to accomplish this it is a pretty serious drain on resources, especially when you factor in the forces needed to man the coastal defences.
This isn't even factoring in the probable (IMO certain) French intervention.
This isn't even factoring in the probable (IMO certain) French intervention.