Japanese Slave Trade Without the Edo Period

IOTL, Portuguese contact with Japan saw a growing Japanese slave trade, as Japan's perennial civil wars and poverty created a supply of captives and desperate people who were sold into slavery, and put to work throughout the Portuguese Empire.

This trade was banned by both the Portuguese and Japanese authorities. The Portuguese and Japanese slave traders ignored the proclamations of the Portuguese government, but the Tokugawa Shogunate's enforcement of isolation ended the trade.

In a scenario where Japan remains open, would the slave trade continue, and would it grow to the point of destabilizing the country? I think it's pretty much a given that this would occur in a destabilized, war-torn Japan, but what about one where a central authority manages to enforce peace while keeping trade with the outside world open? Would they be able to ban the slave trade (if they were so inclined) or would they not be able to control this trade?
 
Under a consolidated but open Japanese state, there's a lot of incentive to shut it all down and enter seclusion: Between the Christians looking like a destabilizing force, Guns and slavery actually being a destabilizing force, and a lot of this traceable to the Europeans arriving on the shores, kicking out all foreigners is a tempting conclusion.

One easy way to keep Japan open is to have the ruling class go Catholic very quickly. The fear that Christians will undermine the state and support foreigners is alleviated. However, this might lead to Buddhist violence as entrenched officials like monks or lesser nobles looking to get up a step stir anti-Catholic sentiment among the local populace. It could easily evolve into a place where the Catholic nobles are warring with and enslaving buddhists. Once the prisoner-to-enslaved pipeline starts to wane, either it'll turn into a Kongo style situation, with slavers preying on the vulnerable of society, or perhaps Japan doing it's own slaving missions north to Hokkaido or Sakhalin.
 
Top