New Orleans feels like a bad idea to place the capital at (as their largest city) as it's on the coast and is susceptible to any naval attack.
With the right fortifications, New Orleans could successfully resist naval attacks as it did in 1815 from the British. Five months earlier in August 1814, the capital of the United States, Washington, D.C. did not, was captured and burned by the British. Still the capital remained at Washington, D.C. because of what it symbolized to Americans.
Other close-to-ocean capital cities like London, St. Petersburg and Tokyo avoided capture for virtually all of their histories.
On the other hand, mere distance from the coast does not guarantee a capital city will not be captured by an enemy. Mexico City, 200 miles from the Gulf of Mexico, was captured twice : by the Americans in 1847 and by the French in 1863. Beijing (Peking), 93 miles from the coast, was occupied in 1900 during the Boxer Rebellion.
Which enemy nation would launch a naval attack on New Orleans in TTL, and most importantly why, has to also be considered. I can only think of three capable of doing it: the United States, the British or the French. Since the United States is right next door and an intensely bitter fratricidal war with them has just ended, they would be the most likely IMO.
The British in TTL would no doubt have been relieved to see an American threat to British Canada cut in half, and greatly lessened, by the splitting up and weakening of the United States. Ditto the French with their imperial ambitions in Mexico. Both nations would have every reason to be pro-Confederate and the South would have every reason to encourage this friendliness since 9 million Southerners are facing the much larger 23 million people in the United States across a long border which is an on-going threat.
Which brings the discussion back to the choice of a capital for a new nation of just 9 million people. They would no doubt want to have their nation taken seriously on the world stage and impress other nations and to try to outshine the United States with its New York City (1,478,103), Philadelphia (674,022), Boston (250,256) and Washington, D.C. (132,000) and all the delights and distractions that they can offer, and impress, foreign visitors.
So why pick an isolated, far inland, tiny town that probably no European at least has ever heard of, as your capital to represent you before the world ? Most of the towns that have been mentioned in this thread as potential capitals have populations that, in OTL, ranged from 3,245 (Opelika) to 4,473 (Charlotte) to 7,401 (Columbus) to 7,426 (Danville) to 10,588 (Montgomery) to 15,389 (Augusta).
Compare this to European capitals in approximately the same time period who were often ignored by the Great Powers. Almost none of these countries had such a tiny town representing them as their capital. Lisbon (210,000), Bucharest (160,000), Amsterdam (279,221), Munich (230,023), Christiania AKA Oslo (151,239), Turin (173,305), Naples (448,743) and Stockholm (133,597).
And the South would not have wanted to be ignored. New Orleans, with its nearly 200,000 population, is the only serious, logical choice for a South that wants to put its best foot forward on the world stage. Other nations would no doubt recognize the Confederate States of America and send diplomatic representatives but I doubt they would hang out for long in an Opelika or a Corinth or a Columbia. They would all flock to New Orleans anyway. Like the song says “How Ya Gonna Keep’em Down on the Farm ( After They’ve Seen Paree) ?”