Yamamoto said exactly what would happen.
They also chose one of the absolute worst places in the U.S. to attack.
Not only is there Camp Pendleton, there is also the MRC San Diego, Camp Mathews, North Island, San Diego Naval Base, Camp Lockett (11th Cav), Camp Callan (6,000 troops, artillery training center) and the local National Guard Armories, plus, as mentioned law enforcement that is extremely well armed (militarized U.S. law enforcement isn't a 21st Century concept) along with a few hundred thousand member of the local gentry with everything from Grandad's Cap & Ball pistols all the way to BAR (civilian hunting variants.
Biggest, and worst, impact will be the massacre of Japanese civilians in the region by locals "huntin' for Japs"
The fact that even the already vastly overconfident Japanese General Staff didn't even consider this sort of thing rather speaks to the concept.
NOTE: It it critical to also consider the Japanese "end game". As ignorant as the General Staff appeared, even the most racist and nationalistic didn't believe that the Empire could prevail in. a war of attrition against the Americans. The goal was a flash knockdown, followed by the establishment of defensive perimeter that would present a barrier that would be too costly for the Anglo-Americans to fight through. The Empire would then be able to negotiate from a position of strength, perhaps pay indemnities for losses and make purchases of territory.
The Japanese were trying to conduct a classic 19th Century Colonial War. Sending suicide troops to slaughter civilians in the enemy homeland simply to do it wasn't part of the game plan.