It's because the populations taken to the Americas are the result of corn, manioc, peanut and sweet potato dramatically increasing the carrying capacity of the land.
I thought the two happened concurrently with the large scale slave trade happening first and then the Columbian Exchange softening the demographic blow so that Sub-Saharan Africa didn't become completely depopulated. I read in "Africa: A Biography of the Continent" that because of the trans-Atlantic slave trade, Africa's population stagnated for a full century while Europe's and Asia's took off. It's too bad that the African leaders didn't use the benefits of the Columbian Exchange to set their people to greater cultivation and metalworking and general expansion instead of demographically plundering each other and their neighbors. Of course, once guns started being traded into the continent on a large scale, I guess that was a point of no return.