The ones that would mainly be skilled laborers from the south and unskilled, but with a higher % of the former. Freed blacks would immigrate as well, and most of those with means would be the small black merchant community (it existed, plus there were assistants to white businessmen who were managers) and skilled laborers, plus anyone who made it to the "middle class", using that term liberally (literate and does clerical work, largely for white masters or black communities).
The result is the arrival of a new ruling class in Haiti and a second society on top of that. The american black won't be seen as the same as the ones in Haiti. Some will be normal laborers but the skilled laborers and small merchants that move over would be come a rich upper class and elite. Hatred for Americans overall, even if Haiti becomes more prosperous, but the US government wouldn't let them be kicked out (where would they go? Not home!). Eventually from this position of security and ties back home to the "free slaves and move them out" movement, the government would slowly be dominated by American blacks and their allies. In the early 1900s historically the black republic becomes prosperous, so it would be even moreso, and the American blacks will add a layer of "sanity" (not intending to be regionalist, just saying that Duvalier wouldn't be in power) that would avoid todays' problems.
As for USA, salve revolts would be bigger from returning emigrants organizing, and the civil war would start slightly sooner.