I also was surprised by how fast it happened, and am not sure of the reality of such a happenstance, but I would counter your idea that it could never happen with such examples as (British) Guyana, Trinidad and Tobago and the Cape Colony (granted Afrikaans stayed a major language).
Trinidad had a small Francophone population of mixed Spanish and French ancestry, and a larger population of free blacks and slaves. But it was still underdeveloped by Caribbean standards at the time, and the original culture was essentially expunged with the huge ramping up of the economy once the British took over.
Guyana was even more clear cut, as British settlers started coming in in the mid 1700s. It was probably already Anglophone when the transfer took place.
As for the latter, I don't think any Boers switched over to English - they just left for the Transvaal and the Orange Free State. And of course, the Cape Coloured didn't ever switch to English.
Going around the world with some other examples:
1. The former Danish West Indies (our U.S. Virgin Islands), were never Danish speaking. A dutch creole used to be spoken, but the islands were never dutch speaking.
2. The Dutch West Indies, despite being Dutch to this day, never went over to speaking Dutch. The "ABC" Islands off of Venezuela all speak a Portuguese-based creole, and the northern islands like Saba, Sint Eustatius, and Sint Maartin (including the French side) all speak English, despite being continually occupied by the Dutch since the end of the Napoleonic era, if not before that.
3. The British have been the best at facilitating language transfer in the Caribbean. Saint Lucia, Saint Vincent, and Grenada all once had French Creole speaking populations, which have now been largely supplanted by English. People in Dominica, in contrast, are still almost entirely bilingual in Creole and English (except in the far north, where some residents from an anglophone island (Saint Kitts IIRC) were settled.
Regardless, in all cases we're talking about creole languages spoken by the lower classes, which might not be the best comparison.
Off the top of my head, the only successful language shifts in settler states where populations being outswamped weren't the reason were in Canada (isolated francophone areas in places like Nova Scotia and Newfoundland), and New Orleans, which didn't see a decline in French until the 1880s.
These were, however, isolated islands within an English culture. Ultimately, the Hispanophone segments of the Caribbean were the earliest settled in the New World. By 1841 Cuba already has over a million people. To put this in perspective, Virginia had 1.2 million at the same time, despite having a much larger land area. By the standards of the time period, Cuba was pretty full.
Now, it is true that Britain seized the islands during a period that the white population was much, much lower (only around 45%, rather than the 66% or so today). Presumably Spanish migration to the area won't happen, and most of those who migrate will be English-speaking. But Anglophone people will have many other places to migrate as well. I'd say a "divided" culture similar to Quebec under British rule is most likely, with an eventual resurgence of the Hispanophones. Similar movements would probably happen in Santo Domingo and Puerto Rico. Haiti will probably shift over to English however to some degree - Creole languages just don't have as much prestige, and they are far more culturally isolated.