Yes, it's smaller and yes, not raising them yet. Khoikhoi's weren't very numerous also.
On the ther hand, sooner or later, they will need slaves, either in Cabo or in Brazil ( and what a nice opportunity to get them in Cabo ), so I don't think that that decision will stay for too long, or too...