After 1848 you could have a group (be it Republicans, Fascists, Monarchists, whoever) take over mainland France, but have the current government (who maintains the loyalty of the navy) flee to Algeria.
Algeria after that point was considered part of metropolitan France, and had a decent number of French colonists and other Europeans living there.