We don't know when exactly with any degree of confidence, but a long time ago.
Modern Japan and the modern Japanese people appear to be more-or-less continuous at least back to the Yamato polity of the Kofun period, with the partial exception of the post-WW2 American occupation of Japan. Yamato probably became dominant around 250 AD, but their origins are lost in the mists of prehistory. It seems to be an open question to what extent the Yamato culture represented a conquest/displacement of the earlier Yayoi culture vs an evolution and political unification. The Yayoi show up in the archaeological record some time between 1000 BC and 300 BC, depending on who you ask. They were probably at least partially a migration from the Asian mainland (most likely Korea), but there seems to be some debate about that, too.
Traditionally, the formation Yamato polity is attributed to the
Emperor Jimmu in 660 BC, but he's a figure of legend on par with King Arthur for England or with Romulus and Remus for the city of Rome.