This is a link to a webpage about the French Empire in World War II.
http://www.worldatwar.net/timeline/france/empire40-45.html
It looks as if the following French colonies had gone over to the Free French by the end of 1940 IOTL
French Equatorial Africa
French Polynesia
French India
New Caledonia
Furthermore, the Vichy Government had to replace the governors of French Somaliland and French Indo-China because they refused to accept the Armistice. And in July, 1940, in Upper Volta (part of French West Africa), "
The colony’s chief administrator Louveau is imprisoned on orders of Boisson after issuing proclamation of adhesion to Free France."
In September, 1940 the Vichy Government sent a squadron of six cruisers from Toulon to restore control in French Equatorial Africa. It passed Gibraltar without interference from the British. However, it was later intercepted by (John) Cunningham who ordered them to Cassablanca. Two complied but the others went to Dakar.
Also in September, 1940,
"St Pierra & Miquelon - Sep-14-1940 St.Pierre - the Ex-Servicemen’s General Assembly of Saint Pierre and Miquelon announces its support for DeGaulle. British Foreign Office sends note to Ottawa urging the Canadian government to support the movement. Canadians decline to act and the islands Vichy governor dissolves the veterans league."
Then in December, 1941
"St Pierre & Miquelon - Dec-24-1941 St. Pierre - Free French Naval Forces take possession of Saint Pierre et Miquelon without firing a shot. Strong protest lodged by American Secretary of State Cordell Hull but after several weeks of bickering between U.S., Canadian and British diplomats and "the so called Free French" as Hull describes them the coup remain fait accompli."
May, 1942,
"May-27 Wallis and Futuna Islands rally to Free French following arrival of the sloop Chevreuil from New Caledonia."
French West Africa seems to have gone over to the Free French in November 1942 as a result of Operation Torch. French Somaliland went over to the Free French in December 1942, but it required an invasion by Free French troops from Ethiopia to do it.
However, French Guiana didn't join Free France until March 1943.
Guadeloupe and Martinique didn't join until June 1943.
The website isn't clear, but it seems that the last Vichy force to join Free France was the naval squadron at Alexandria, which also happened in June 1943.
***************************************************
My guess is that all the French colonies that joined Free France in 1940 IOTL would do the same ITTL.
My guess is that the colonies that would be solidly Vichy ITTL are French North Africa, French West Africa and Syria, which are the ones that count, plus Guadeloupe and Martinique.