Having Manhattan as the district would've made it very small. The original DC formed a square that was 100 miles^2, before retrocession of the Western part to Virginia. Manhattan only is 23 square miles. New York as the capital probably would've also taken in the Bronx, Queens, Brooklyn, and Bergen Township, NJ (which IOTL became Hudson County)
I admit that I thought of the size difference later.
But, given that Manhattan wasn't certainly fully filled at the end of the 18th century, it would not have been considered sufficient at the time and then successive additions to be granted? I mean the District even gave back the Maryland segment and well the population of the District today isn't comparable to Manhattan...