I’m not sure it’s quite possible to have Brazil not be a world power without a PoD in the 1600s. Their population is so massive, their mineral and agricultural wealth so vast, their dominance of South America and later the Southern Hemisphere seems inevitable imo.
I think that if you just want to significantly nerf them, the easiest answer is to have them loose the Cisplantine War. By winning and securing Cisplantina, they were forced to build a powerful martial tradition to defend against the Paraguayan-Argentinian Alliance. And due to that tradition, they were easily able to conquer Paraguay and seize dominance of the Rio de la Plata, which made them even richer, and allowed for even more internal and external development.
If they loose the war and then waste resources fighting future wars with Argentina, they would be significantly poorer. Add in Britain or France exploiting this weakness to leverage influence on Brazil, and maybe a Republican Coup or more powerful slave revolts, and you could have a halfbaked power. But I still feel like eventually they would go through the industrial revolution and come up to dominate South America at the very least.
As a side note, Bolivia would likely be far less rich in such a timeline. Their alliance with Brazil was founded on their mutual disputes with Paraguay, in such a timeline it might not exist if Brazil is focused on Argentina. And their wealth mostly originates from the Brazilian transcontintel railings running through their country. That wouldn’t exist without the Brazilians conquering Paraguay. Maybe ITTL the tech and trade hub of the South America is somewhere else? Perhaps Argentina if they’re more successful?