Even without the Holocaust, you're still looking at a regime that is undemocratic and totalitarian, makes ample use of political violence towards dissidents, oppresses its citizens it deems "undesirable" for bigoted reasons, wages wars of expansion against its neighbours for irredentist reasons, and so on. It probably wouldn't have the reputation as being one of the evilest regimes in all of history, but it's hard to see anything other than a strongly negative legacy. Hitler wouldn't be a Napoleon.
So, maybe a far-right counterpart to Stalinist Russia in those ways, then?