Japan was open to the idea of splitting the region, so its influence would dominate in Korea and Russia's in Manchuria. It was not unlike the earlier agreement that saw Japan drop claims to Sahklin Island in return for Russia dropping its claims to the Kuriles. However, Russia saw no need to reach any agreement with what they assume (largely due to racist reasons) was a much weaker power. Some of the Tsar's advisors also favored a war with Japan precisely because they assumed it would be a quick, victorious war, which would add to Russia's prestige.
As for Japan, they had a longstanding policy that Korea could not be allowed to fall under the influence of a potential enemy, given its proximity to Japan. Thus the famous phrase, "Korea is a dagger aimed at the heart of Japan". This was the cause of the First Sino-Japanese War, as well--the main result of the war was that Qing China was forced to recognie Korea as an independent state. After shedding so much blood, Japan will never allow Russian influence to replace Chinese.
The war might be avoided if Japan is reasonably convinced that Korea won't be used as a base against them. As Japanese confidence grows, more and more Japanese want to take over Korea outright. However, if a neutral or Japanese-influenced Korea is all Japan can get without war, then they will take it.
Russia is harder. They are convinced that the richest of East Asia are theirs for the taking. What's more, they think expansion here will help prevent or solve problems at home. I don't know what you can do to convince them otherwise. Perhaps if Russia is otherwise occupied, by expanding into Mongolia or China?