The Nazi party was one of, if not the worst organization in the history of mankind. The kind of "weeding out" you refer to would make Stalin's purges look moderate by comparison.
Not just worst in terms of morality(though it was certainly an organization of evil, and not the fun kind), the Nazi power structure was deliberately and literally designed to be as corrupt and inefficient as possible. If you think Nazi military government was incompetent*, just wait until you see Nazi civilian administration!
*This isn't to say a non-incompetent German government could have won the war, at least not the "GERMANIA FROM TEH ATLANTIC TO TEH URUHLS!!11!1!!1! fantasy the Nazis ascribed to.
More than that though, when you consider who those who operated in the highest echelons of power in the Nazi state a purge of their political enemies is more likely to get rid of the sane people (you know, the ones on the level of a typical serial killer) than the really crazy ones (Heinrich Himmler).
Hey, Himmler was only one or two standard deviations more horrible than the average serial killer. Heydrich was the really nasty one.