Main challenge is that the territory is gutted. The USA will look the other way as they slit the throats of the German settlers the Nazis planned to move into the area, but rebuilding the agriculture and population will take a long time.
Really depends on when it happens. If the Reich collapses within a decade of the war with the Soviets concluding - say, Germany gets a quick and decisive victory against the USSR in 1941, but Hitler still gets into a war with the United States which ends with several major German cities getting nuked - then the rump USSR will fairly easily reabsorb its European territories only a few years after losing them.
But if the Reich collapses, or at least retracts, decades after the end of the war, then at least the Baltic, Belarus, and the Ukraine (with Crimea) will be probably be thoroughly Germanized, both by settlement and assimilation. Reabsorbing this territory will be much more difficult for the Soviets - not least because a USSR that has spent decades shorn of its European territories, or even “merely” having lost an amount of territory similar to that stipulated by Brest-Litovsk, will be a fairly weak state. Plus, regaining the Caucasus is probably a non-starter in this scenario anyway, as the Armenians, Azeris, and all the rest are
not going to happily trade one occupier for the other, and then US will likely be happily supporting them as the Reich collapses.