my general impression is that the Japanese then really didn't step back from doing something just because it was stupid.
LOL. Bravo.
their Empire building ambitions
It wasn't entirely that. There was a goal of achieving
autarky (&, I have a sense, of returning to pre-Meiji isolationism).
If the Japanese government had serious considerations about a compromise like that phx1138 describes above I think it was the fall of France in June 1940 that weighed decisively in on the "go all out" option as it gave a realistic opportunity to seize FIC, which again was necessary if Malaya, Singapore and DIC was to be taken.
I can't say as a fact it was ever seriously considered. The fall of France, & the weakness of Britain, did seem to tip the balance in favor of invading, with a sense they'd be unable to respond effectively.
Next question is what happens if France still falls, but FIC is in the camp of de Gaulle and not Vichy. IMHO it would still be too stupid to attack.
Now you've got a crux. Japan is going to see French weakness, combined with opportunity & a real need to cut supply lines to Chiang's forces in southern China. Is it a gamble? Yes. Will all the parties opposing Japan be stronger later? Yes. Does that make it worth doing now, rather than later? IMO, there's a really good chance Japan says yes to that, too.
it would mean a full involvement of the British Empire (and probably USA too)
I'd agree on the first, but not the second. In 1940, it's less evident in Japan the U.S. & Britain are "joined at the hip". So there's a reasonable chance, moreso than 1941, Japan leaves the U.S. alone. (Unless you believe IJN posited the attack on Hawaii to avoid becoming the poor stepchild in defense $$ allocations...which may well be the case.) That being so, it's just possible you don't see attacks on Hawaii or the P.I. TTL.
That makes it impossible to achieve the actual object - gaining access to resources in Malaya and DEI.
Yet that obtained OTL, & didn't dissuade. There's really no way Japanese planners could know the outcome would be as bad as it was--& given their
penchant for rosy predictions...
They had a binary choice - find some way to end the war in China through diplomacy or seize the resources needed to continue the war.
That suggests U.S. diplomacy really was pretty inept, since it managed to push Japan in exactly the opposite of the desired direction. I wonder about the U.S. expressly offering Japan a deal: cheap oil for 5yr, in exchange for leaving metropolitan China (& keeping Manchuria), & trying to get Chiang to sign off on it, too.