Not at all. At most, you could have a "Japantown" on par with OTL San Francisco's Chinatown, which is massive and has had a big influence on the city (and the Bay Area in general). But for a single immigrant group to completely dominate a US city to that extent in the early 19th century is just ASB.
A much more plausible version of this would be Honolulu. It already has the largest Japanese-American population in the country, and it's a lot smaller than San Francisco. So I think it's well within the realm of plausibility for Japanese immigration to transform the city in the way that you're describing. Just find a PoD that keeps Japan from attacking the US in 1941. Japan exhausts itself in China, the Japanese economy collapses, huge numbers of Japanese immigrants start heading to Hawaii to seek a better life. By the late 20th century, when Hawaii finally becomes a state, Honolulu is to Japanese-Americans what Miami is to Cuban-Americans.
It would also help if it were legal for Japanese people to immigrate to the US and naturalize there. In OTL, immigration of Japanese men was stopped by the so-called "gentleman's agreement" between the governments of Japan and the US. The Naturalization Act of 1906 provided for naturalization only for white people, ; in 1922 the US Supreme Court found in Ozawa vs. US that this act made Japanese people specifically ineligible for naturalization. (It is important to note that only Japanese naturalization was banned. Japanese-American children born in the US were citizens by birth, so that the majority of those interned during the war were in fact US citizens, though a goodly number of internees were first-generation immigrants banned from citizenship). The Immigration Act of 1924 specifically banned the immigration of those ineligible for naturalization, thus totally banning immigration from Japan, China, etc.
In addition to these federal measures, there were state laws in areas with relatively more Japanese immigration. For example, in 1913 California passed a law banning Japanese people from owning land in that state. Asian and Asian-American children were forced to attend segregated schools in San Francisco from 1906. Japanese and Japanese-Americans protested the segregation order (previous attempts to segregate the schools had been successfully defeated by protests), leading to a period of high tension in the city. As a result 1907 saw three seperate race riots targeting Asians in San Francisco (these riots inspired similar riots in Bellamy, Washington and Vancouver, B.C. that same year). The first riots were lead by specifically anti-Japanese activists, although in many cases the rioters targeted Asian people generally.
The reason all of this started to change was the war. It did not look good for the US to have such discriminatory laws directed against Asians when it was fighting openly racial supremacist powers. In your scenario, where the war is avoided, I assume change would come much more slowly. Therefore, no matter how many Japanese people want to immigrate to the US, it will probably remain impossible for them to do so.
