I was shocked to hear this too...
http://en.wikipedia.org/wiki/Black_people_in_Nazi_Germany
Just read the very first sentence of the article.
However they obviously didn't gel well with the Aryan race ideology.
I did read somewhere that a lot of black Namibians moved to Germany proper in the interwar era. Some of them became involved with white German women. IIRC there is a passage in Mein Kampf where Hitler comments on this, talking a bit about rape, and defilement of the Aryan race; stuff like that. So Hitler wasn't too fond of black people, but I suppose the Nazis didn't see it worthwhile going out of their way to persecute them on the same level as Jews communists or Slavs.