There's an few issues with this.
You have a population loyal to the Crown but it's a small population that's situated on the lowest tip of Manhattan Island which in the late 1700s was mostly uninhabitated north of Canal Street save for a farm or two. (And Dutch). Maintaining control over New York without getting the Hudson valley is tenuous at best. And New York isn't a natural fortress to boot.
Tariffs would also make NYC a poor place to serve as an entrepot and the city would suffer greatly.
NYC and Long Island in this case would be an sort of ealry Hong Kong, with the fact America could storm the city and Long Island with ease and the British not being able to do much.
OTL, the British were very generous to the US in the Treaty of Paris, as they wanted to woo the US away from their French alliance. Keeping New York City would go against that. (The end result the UK got far more from an independent and friendly America. This would only invite a future war where the US will attack and conquer the city.)
HOWEVER.
Loyalist groups ranged as far north as Westchester County, which could give them some strategic depth. A British New York would be massively influential. A second harbor next to Halifax in British North America, easy access to their Caribbean colonies, access up the Hudson River, and a natural port for refueling Royal Navy vessels.
The Caribbean would also help enrich New York/Long Island. It's a shorter distance than the Caribbean and England, plus Royal Navy vessels would find a friendly harbor where they can easily resupply.
(I don't think the British could, or want to hold onto New York/Long Island, I am just showing the other side of it.)
Could also keep Philly as the capital which is always an win.