A huge part. The Japanese had never forgotten the way that Japanese immigrants were treated at the turn of the Century (laws limiting Japanese immigration were, even for the era, quite restrictive, and Japanese immigrants were not allowed to become citizens).
The racial divide went both ways...