Louisiana isn't the poorest state, at least not by certain important metrics. California has a higher poverty and near-poverty rate, a higher per capita homeless rate, and worse inequality last time I checked. Mississippi has worse educational outcomes, and states like California aren't too far off from Louisiana. New Mexico has a higher percentage of people on food stamps. Plus as others have pointed out the median income in Louisiana is actually fairly high.
The problem with New Orleans specifically, however, is just that; it's economy is really structured around a few parties/celebrations a year that draw in thousands of people, and that's it. It never had substantial industry like Buffalo or Chicago, but it was an important financial hub in the South for a while. Then the post-WW2 boom happened, and suddenly many other Southern cities that were struggling after the Civil War/Depression began expanding, and places like Charlotte, Atlanta, Miami, Dallas, and Nashville became more important financial hubs. It also didn't help that the decline of manufacturing saw a drop in the need for New Orleans' river traffic, and the city had a notorious flood/storm problem.
Plus, frankly, New Orleans hasn't had the best reputation among Southern cities. Many of the people moving to the Sun Belt are elderly and families, and New Orleans just isn't seen as family-friendly or elderly-friendly as, say, Murfreesboro (a suburb of Nashville). It has a fairly high tax burden compared to other Southern locales, and for the longest time had a much higher/visible crime rate and corruption problem. Plus it has a reputation for being the land of drunken frat kids, druggies, and other kinds of people families/elderly tend to avoid. Katrina certainly didn't help, but the city had been declining for years before, and there's still a lot of damage left behind.