First off, thanks for teaching me something new. Stuff like this is why I like AH.comPortugal controlled sugar production since the 15th century which expanded throughout its empire, firstly to Madeira, later to the islands of the Guinea Gulf and finally to Brazil, pretty much creating the concept of triangular trade. However, Portugal was deeply dependent on Flemish/Dutch merchants to distribute their exotic goods in Northern Europe, and, with the union of the Iberian crowns after the sudden death of King Sebastian in Morocco, the Portuguese empire entered the disastrous Dutch Independence War and disrupted their mutually advantageous commercial relationship.
The war was, obviously, disastrous to Portuguese economy, as the Dutch actively tried to conquer Portuguese colonies, Portugal then lost most of their empire in the East and almost lost most of their Atlantic empire when the Dutch invaded Brazil and a good number of West African slave ports. Of course, the conflict made the price of sugar skyrocket and prompted the creation of sugar plantations elsewhere (see that first Caribbean colonies were founded by the 1620s/1630s). The final blow to Brazil'sugar cycle was the restoration of Portuguese rule over Brazil (to be fair, Catholic Luso-Brazilians took back Brazil with little help from Portugal) and the Dutch and the Sephardic Jews took their money and newly acquired know-how to the Guianas and the Caribbean.
Second, I think we now have our POD: King of Sebastian does not die. Portugal does not get shotgun married to Spain and does not get as deeply involved in the Dutch conflict. I am not sure they would be utterly left on the sidelines of the Dutch-Spanish conflict, but lets just say it does not go as badly for Portugal and as good for the price of sugar (somewhere Homer Simpson nods along to all of this, for as he knew all long, "first, you get the sugar"). This means the initial economic impetus for the colonization and etc. of the Caribbean is not there. That means, slavery is not economically viable in the Caribbean, or at least not as economically viable as early as it was in OTL, due to there not being much profit in importing slaves to raise a cash crop that is now no longer yields as much revenue. Still, I'd imagine someone will want to take those islands and do something with them, just not something as labor intensive as sugar cultivation, and therefore we downgrade the volume of the flow of slaves into the Caribbean. And now the butterflies are let loose.