What is attracting Americans to Baja over Alta California?
The same reason some people chose to move to Los Angeles and San Diego instead of the Bay Area.
There were also several gold rushes between 1850-1890 IOTL. That will attract settlers as well.
Even in the southern portion of the penninsula you are going to have merchants and fishermen set up shop.
There are more than enough immigrants and US population grown that will cause some people to settle in an American controlled Baja penninsula. There is nothing magical about it that would prevent people from showing up, and alot of them are going to stay.