The period of deindustrialisation in America would probably be a 'good' window for anti-Asian to flair up and begin a habit of denouncing martial arts schools. As manufacturing moved to new Japanese factories, there would likely be a lot of people in the Rust Belt left behind to look at martial arts classes run by Japanese immigrants as a part of some fifth column to indoctrinate America's youth for eventual Japanese economic and cultural domination.
Interesting scenario. It might help if the Japanese government maintained foreign schools akin to China's Confucius Institutes or the UK's British Council, and if those schools taught Judo or Karate. Rather than simply free-standing martial-arts schools. Then, it would be easier to convince people that martial-arts were being directly promoted by the same people who were depleting the Rust Belt.