As we know, Hitler was terrified by the idea of the two-front war, which he considered a main cuplrit for Germany's FWW defeat (along with the Judeo-Bolshevik cabal, evidently). We know Stalin kept repeatedly delaying signing the pact (having already been informed of the invasion's potential starting dates), so as to force Hitler accept his conditions wholesale (he was simultaneously playing the Entente, obviously without any desire to sign a mutual assistance pact, but fully conscious that the Anglo-French will not back down in this instance, like they did with Czechoslovakia).
What if Hitler's inveterate gambler instinct had broken his fear and made him say "Fuck it, we're going in regardless"? Stalin was utterly contemptuous of the Entente and had 0 desire to, in his words, "pull out British and French chestnuts out of the fire with Russian hands". He was massively paranoid of ending up played by the Entente, which is why he only issued the invasion order for Eastern Poland after the Germans had already reached, and stopped at, the aforementioned agreed-upon partition line. We also know that the economic portion of the pact gave the Germans precious little of true hard value; they had already managed to strongarm Romania and others into heavily one-sided agreements, and Stalin had the habit of intentionally delaying some of the shipments (in a way that would make it seem like a random act of underling incompetence; he was great at this sort of stuff). Stalin, OTOH, got the whole damn Old Empire back. The discrepancy in value is readily obvious.
Again, what if Hitler's gambling addiction and the Russophobe wing of his acolytes combine to say "Jacta alea esto!"?