Because the agreement was that some land would be given to the Soviet Union to give them year round aces to the Baltic. They were keeping up the polite fiction that the soviets hadn't went through with their illegal annexation of the Baltic states and that they weren't going to rig Polish elections. Stalin may have been brutal, but he was clever. Besides, the Poles would have slaughtered or driven out the East Germans, if only because- Actually, do we really need to explain? The Poles are not keeping them.