But would they want to stay in Brazil? If look at the Confederate immigration, it actually can be considered a failure. Only one of their colonies survived, and maybe half of them went back to the USA. It happened basically because the Confederates wanted to recreate in Brazil their lifestyle, they wanted to be a slaveholder upper class. However, when they came here they discovered that Brazilian slaves were much more expensive than they could afford after the war, and they simply didn't have the money to buy all the men they needed to run their farms. Also, many in São Paulo didn't want them as new "elite" in distant regions, and some farmers even thought to use the Confederate immigrants as a new source of workers to replace the already scarce slaves.
The only colony that prospered was the one in Santa Barbara do Oeste, where today is located the city of Americana. And that only happened because it was the only place where the few Confederates could find cheap lands and at the same time buy slaves to run their farms. And even there some were forced to live as small farmers, using their own family workforce to survive. And this was the reason why so all the other colonies failed. When people who had never worked by their own hands were forced to do it they simply gave up and went back to America.
An interesting example can be found in the colony of Juparanã, in the province of Espirito Santo. One immigrant, Josephine Foster, commented about the fact of her father, a 57 years old man who had never made hard work in his life, was forced to do it in order to survive: “...mainly by his own exertions, has cleared four acres of land, two of which he has already planted with corn, beans, sweet potatoes, mandioca.” And she says about the reason to the settlers go back to the USA: “Everything they have said in praise of Brazil and Brazilians, seems to fade away in the distance when they are thoroughly convinced that manual labor is the one thing needful to help us on to a fortune in this country.”
So, if you have much more Southerner immigrants you only would have more deluded people in Brazil. The price of slaves would become even more expensive due to the demand, and fewer of them would afford it. In the end, they would be forced to go back to America, find urban jobs (as some of them did IOTL) or endure a poorer lifestyle than they were used to in the South.