I located the relevant Wikipedia page on the subject:
https://en.wikipedia.org/wiki/Residence_Act
It seems from the article that the decision to move the capital to what is now Washington was taken quite early, in 1790. And everyone seemed determined for the capital to be some city constructed out in the sticks, the debate being over which side of the Mason-Dixon Line it was located in.
Reading between the lines, it seems that the politicians thought that the inhabitants (mobs) of Philadelphia were taking too much interest in their doings, and wanted to get the capital away from a city as soon as possible. It wound up in New York basically because they were trying to get away from Philadelphia.
This is a pretty ingrained cultural attitude in the US. Look at where most of the state capitals are located.
The thing is, if they decide they are OK with keeping the capital in the largest city, the largest city in the US in the 1790s was Philadelphia, not New York. I can see a POD involving the government not fleeing from Philadelphia. I don't see anything that would make New York anything other than the temporary capital. Even after New York surpassed Philadelphia as the largest city, Philadelphia remained the second largest city for quite some time, I think about a century, and is still a large city. And if the federal capital had stayed there it would be larger and more important than it now is by virtue of that fact.
I can see Philadelphia become Rio to New York's Sao Paulo, though of course the Brazilians moved their own capital to a planned city in the sticks.
One problem with keeping lots federal government offices in lower Manhattan is that that area is badly laid out and really crowded. Old City in Philadelphia doesn't have the problem, and there is also Camden right across the river. One unfortunate consequence of keeping the federal capitol there would potentially be the destruction of lots of historic buildings.