I seem to recall reading somewhere that the capital was originally planned to be further up the Potomac (Georgetown would be Southeast DC in that scenario). The somewhere may have been a Gore Vidal essay, so grain of salt. New York was the seat of the federal gov't while DC was being built, so if it never gets built I think that New York would remain the capital out of inertia.
The Southern states were very much insistent on a capital located in the South, and any such location would have to be accessible to the rest of the country. I therefore think that a spot on the Potomac was very likely. Plausible alternatives to its current location would be the slightly upriver location I mention above, or perhaps a Virginia location across the river. The latter option presents some difficulties. It would increase the sense of Virginian domination during the 'Virginia dynasty', and it puts the feds in a rotten position should something like OTL civil war occur.