The USA could not do anything to Brazil. In fact, in several periods, the Brazilian fleet could have completely destroyed the US Navy, and invading Brazil is beyond XIX century's USA capabilities. So that's the easy part.
Now, annexing South America? Brazil could attempt an invasion of the Southern Cone, probably take Paraguay, Uruguay and even Buenos Aires. But they would quickly get bogged down in the occupation. If you read about South American history, gaucho armies rise up very easily, and are hard to defeat. They would live out of the land and constantly strike down at Brazilian armies and their local allies, and probably erupt in full-blown rebellion when Brazilian armies are distracted. The terrain in Argentina is also hard beyond the Pampas, with dry valleys, dense dry forests occupied by 'indomitable indians' (and they were called that for a reason) and hills. Unless Brazil is ready to fully commit its resources to occupation or engange in enthnic cleansing, I don't see them controlling effectively anything beyond the Paraná river in Argentina, perhaps only by proxies and friendly caudillos (and knowing the independence of caudillos, that is very doubtful.
I do not see any plausible way for Brazil to invade Chile or the countries of the Altiplano, even if they somehow got there, the terrain is even harsher. Invading Colombia and Venezuela would mean naval invasions or crossing the Amazon; one hard, the other impossible.
Brazil could be a continental hegemon and install friendly regimes all over (in fact, it did intervene heavily in Argentina's civil wars), and it could potentially invade the Southern Cone, but it would be extremely costly and would be harassed at every turn. Eventually it would be forced to let it go or engage in outright colonization. None of those prospects look very good for Argentina & co. but neither for an overstreched Brazil.