Have to do that incrementally. OTL there was a program to replace PoW with paid labor. There were clear advantages in long term productivity, but the nazis & German business in 1940-41 were focused of short term gains and failed again to organize properly another useful program. Returning some of that labor to work in French factories would work better than the OTL 'system' of hauling trainloads of machine tools to Germany without plan, & then letting them out in storage parks for weeks, months, and some cases the entire war.
In 1942 which failed miserably, with the Germans sending home old and sick labourers in exchange for young and healthy ones at a rate of 1 POW for 3 workers.
The problem you have to address is this.
The POW are used generally in labouring jobs in agriculture, mining, Which have to be done. At the same time - 40-41 the German army is mobilising an additional 58 Inf Divs. for Barbarossa plus all the rest thats just the wave system divisions, the need is manpower, after all one good kick and the whole rotten edifice will come crashing down, home before the leaves fall.
Returning labour to French factories in general is irrelevant at this time as it means you cannot mobilise German manpower for the army, the plant of use to the war effort is the plant where the French would have retained the workforce in place anyway . The economic blockage in general is the production of coal and steel ( and before that calories to cut coal and timber for pit props). It only matters if there is sufficient in the supply chain to begin with that its piling up because the Reich Factories cannot process it, Not the case there is a perennial shortage of steel not factory capacity and the main producing regions for that are in occupied France not Vichy.
And ofc there is a long term plan, which is to deindustrialise France its only in 42 after the failure of Barbarossa that Hitler makes any effort to use satellite industrial capacity, and then he is in a war with Britain, the USSR and USA, which for France means the more the factories are in France the closer they are to Allied bombing the more air defence they need and as Reich air defence at least in part was from factory workers manning AA positions which is a bad idea in France the more German troops have to be sent to France, all around easier to move the plant to Germany especially as in 42 the factory space built earlier is coming on line and needs tooling up.