In theory yes, the question is why Korea would go fight from province to province against the Japanese, in the first place. The Muromachi period was chaotic enough that individual clans basically became trading intermediaries with foreign nations or sent their own delegations. For Korea, this was the So clan of Tsushima, who the Korean did fight, but going after anyone else would be pointless.
Yeah, the Joseon didn't really think too highly of the Japanese and their capabilities at the time (the result of which could be seen in the Joseon's disastrous lack of preparations in 1592). They didn't really put a conflict with Japan as a major concern; the Jurchen to the north were the main problem and where most of the military focused. As for resources, Korea's right next to China, which takes care of most resource needs, and war for conquest's sake wasn't big in the Confucian order, where bureaucrats and studying the classics took priority over commanders and military text. The Joseon were also mindful of military regimes, seeing the failure of the Goryeo and the military dictatorship that held power there for decades.
Though I'm doubtful as to whether the Joseon actually had the ability to go over to Japan in force, seeing the chronic incompetence by so many Joseon commanders in the Imjin War. The military and navy would have to be reformed well in advance to continue to hold the territory once the Japanese begin consolidating.
Perhaps if Korea conquers/gets conquered by Manchuria they'll have the necessary resources and manpower to take on Japan one-on-one? The later such a war happens, it seems to me, the better.
Manchuria wasn't very populous at the time and it'd be more a resource sink, considering raids from nomads and the Jurchen tribes were still a concern for both the Ming and Joseon. Pacifying the region, keeping the Ming happy (the Ming wanted to arbitrate between the Joseon and Jurchen to keep balance in the region), and settling it would be a long, expensive endeavor. On the flip side, the Manchu had no naval tradition, which let Koxinga run rampant on the Chinese coast. And again, China's just a much more worthwhile target for a would-be conqueror. Japan, by comparison, was just not worth the effort.
The Joseon would need a better navy, better officers, and the drive to go ahead and invade Japan while it's divided. The lack of precedent (the previous invasions by joint Yuan-Goryeo forces proved tremendous wastes of men and money), lack of obvious payout, and difficulty of administration (Korea's terrain is hard enough for travel and bureaucracy to be difficult in the north and southeast, adding seas and more mountains doesn't help), among other things, made it not much of a thought in the Joseon court's mind, I would assume.