According to this website "Movements to boycott the 1936 Berlin Olympics surfaced in the United States, Great Britain, France, Sweden, Czechoslovakia, and the Netherlands. "
If these countries didn't go, what would change besides no famous Jesse Owens.