They could've put it at the original Jamestown site.
Only if they built it on rafts. The river has meandered over the actual site, the Jamestown that one can visit today is a replica.
Perhaps if Zorqal would say why the precise spot chosen OTL is one he wants to avoid, we'd be better fixed to suggest other locations?
It sounds to me like the OTL site was intelligently chosen as far as having a central location goes. It is unfortunately below, rather than straddling, the Maxon-Dixon line which was pretty awkward during the Civil War, so I'd go with a more northerly site, say straddling the MD-PN border. But the farther north you go, the less conveniently the site is served by the Chesapeake Bay system.
Perhaps, as with Brasilia, the site should have been placed far into the west, to anticipate expansion of settlement and to draw it.
But before anyone conceived of railroads, this meant dooming the capital to be very difficult to access from any established settlements, slow communications, and a major effort digging canals or even making the sorts of crude roads that were expected in this day. It is not just that the site would be more distant from seashores and inland beyond the currently developed zone--it would be up in the foothills of the Appalachians, in inherently rough terrain.
Or they could have been very very bold and ambitious and set the new capital west of that range, on the eastern sheaf of rivers feeding into the Mississippi. I'm guessing such a Federal District would be in Ohio or Kentucky. And it would pose really severe communications problems for everyone, with westward settlement not really justifying it until the 1840s or so.
Actually, I believe DC as selected OTL was intended precisely to be a gateway to the west. Sea traffic could reach it, and thence upriver leads to passes to the Ohio country. As things turned out, New York state became the major gateway, and indeed NYC was the temporary capital while Washington city was being laid out. But it was too far north for Southern states to be content with it. The far northwest end of the Chesapeake was therefore it. This means that any plausible DC would be within a few hundred miles at most of the OTL chosen site.
To repeat then, why exactly is the current site less than optimal, given realistic constraints of the day? What's wrong with the site chosen? It's not like some other great urban center has risen in that region to show the way. Insofar as the belief and hope that DC would grow into a major commercial center as well as being the Federal capital was mistaken, the mistake was on a larger scale than this or that river mouth; the natural center would be New York or Buffalo, and those are too far north for the Southerners to accept; Ohio is too far west to work for at least a generation, and south of OTL DC is too far south and preempted by existing associated with established states.