How would the Germans move enough men to hold Madagascar? They were too tied up at home to even consider an operation of that distance, and HMS Ramillies and her 15" guns were there to counter shore fortifications. The Vichy French were outnumbred 2-1, and German troops there would guarantee that the South Africans turn up in numbers, too.
The Germans did not have enough amphibious capacity to do landings far from home, and trying to land in South Africa, which at the time was a key part of the British Empire, woulda been impossible. Afrikaners who assisted or tried to assist the Nazis would find the South African PM, Field Marshal Jan Smuts, rather displeased with that, and the South African Army if the Afrikaners rose up as a group could call on British, Canadian or Australian reinforcements, or even bring in black regiments.