Logistically and strategically speaking for the British, this would be a disastrous move. New York City and its resources and location made it the ideal North American base for them. It had access to the Hudson, an overland capacity to threaten Philadelphia, and could do a joint campaign with British Canada. Virginia and Maryland had ports, sure, but that was about it. The Americans would control the richest and most populous parts of the colonies outside of Virginia almost unopposed. There is also the fact that support for Loyalism in Maryland was almost dead and in Virginia, it was limited to the High Anglican coastal tidewater areas. There would be hostile militia constantly harassing any attempts at movement (and this is not a joke of a threat; the New Jersey militia was a massive pest to the British throughout the war, and there was a reason that they never made a single attempt to penetrate deep inside New England). The Continental Army could afford to put less men in the Albany region simply because of a lack of a true threat from the North, and the Army would be able to swell to a large force.
The only upside I can think of is that it would force the Continental Army out of their OTL entrenchments in the Hudson Valley and Morristown areas and onto more open ground, which would benefit the British. But that was no guarantee of success, and any attempts at marching on Philadelphia would be opposed falling back river by river with harassment on the whole way, and any attempts at marching southwards would leave supply ports in danger.
As for effects on slavery, it probably would lead to more contraband labourers for the British in their coastal areas (and many would likely be deported to the Caribbean due to food constraints), but that would probably be about it.