So this is a challenge and a related question-
Could Korea and Japan have launched into large scale expansion without finishing the other one off first? In OTL, Korea never expanded beyond its peninsula politically, and in its OTL historic bids to expand, Japan started in Korea. When they failed in Korea they quit. When they succeeded 300 years later they went after additional targets subsequently.
Southern Manchuria was part of Korea proper (both culturally and politically) for at least two millennia until 926, when Balhae collapsed due to a catastrophic eruption on Baekdu Mountain. Goryeo also ventured into Manchuria and even managed to briefly hold Liaodong and some portions of Southern Manchuria during the 14th century after refugees flooded in. However, no Korean state ever engaged in extensive wars of expansion into "foreign" territory, as doing so would have been suicidal after depleting its resources, not to mention confronting much larger empires within China and Central Asia. Goguryeo and Goryeo were the only exceptions, both of which directly confronted China, although they collapsed within a few decades, as a succession dispute tore the former apart, while Yi Seong-gye decided to stage a coup against Goryeo and found Joseon after disobeying the order to invade the Ming. In any case, neither ventured far past their possessions at the time because cooler heads eventually prevailed, and the succeeding dynasties eventually agreed to pay tribute to China, although Silla temporarily suspended relations with the Tang for 50 years due to the Silla-Tang War that broke out soon after Goguryeo's collapse.
However, the only Korean state that could have managed to retain Manchuria permanently was Goguryeo, which failed to unify the peninsula for over 300 years due to the Baekje-Silla-Gaya alliance holding out for over a century. I'm currently tackling this in my TL, although it's taking a while because Goguryeo has to hold off simultaneous invasions from both China and the southern alliance for close to a century by taking advantage of its adversaries' political and social issues, which is easier said than done.
Could we have plausibly had a situation in the last two-thousand years where both expand simultaneously. This could come in any flavor you'd like. The first that comes to mind is the Atlantic model - the European Atlantic states England, Netherlands, France, Spain, Portugal, checked their ability to take over each other, but managed at the same time to acquire vast overseas territories. The other model is splitting the land and sea domains, wherein Japan expands mostly to the south to additional island or overseas territories, and Korea ends up expanding on the Asian mainland, ruling Manchuria, at least invading China and Russia, and possibly succeeding in holding expanded realm there for several decades.
I'm already planning on a much larger Korea in my scenario, although a similar situation might not necessarily apply to Japan because I'm considering how the Emishi/Ainu can hold out in Hokkaido and Northern Honshu, forcing Japan to focus more on the north. To be brief, Korea will briefly control most of East, Central, and Southeast Asia at its height, although it will then be limited to Manchuria and the Korean Peninsula, as both had been culturally Korean since around 1000 BC. However, this scenario only works due to taking advantage of extreme fragmentation within East Asia at the time, which would only have applied from 390-430, while Korea's expansion into China would not be a requirement.
Have Korea be unified by Koguryo or successor states thereof, and have it survive the coming of the steppe peoples intact. Do that and Korea will have a relatively culturally homogeneous population from the Korean peninsula and the Manchurian plains. Such a state could avoid the under-population & lack of commerce problem that Korea faced due to having a predominantly mountainous terrain, and thus be much stronger than OTL Korea. This could possibly lead to proportionately larger continental ambitions.
For Japan the Tokugawa Shogunate could be butterflied away, with Japan eventually coming under the sway of one of the western clans, say the Shimazu for example. Such a Japan would be much more open to trade and contact with overseas polities, and could possibly have their historic ambitions tied up with the Philippines & the Spice Islands, rather than Korea & China.
I'm currently addressing Korea in my TL, although if Japan manages to expand south, it would take much longer to consolidate its possessions due to the logistics involved.