It's more amazing that they didn't colonise North America outside of the Danish Virgin Islands and technically Greenland. My guess is they got too bogged down in wars with Sweden. Greenland is an interesting case, since German explorer Didrik Pining (who was sailing on the orders of the Danish king) visited it (and may have visited Newfoundland or Labrador) in the 1470s. While the Greenlandic Norse had abandoned their colony by that point and farming was all but impossible, Greenland would make a perfectly good colony for sealing, whaling, fishing, and whatever trade with the Inuit you might conduct. So perhaps Denmark establishes a base there in the late 15th century for that purpose. From there, Pining might venture to Newfoundland before Cabot around 1490 and perhaps discover the Grand Banks where Denmark will take control of the country using their claim on Greenland and Vinland for legitimacy.
By the early 16th century, Denmark should have settlements in Newfoundland and perhaps have charted most of the Eastern seaboard. OTL the admiral Søren Norby was told by the king of Denmark Christian II to find a way to the West Indies by way of Greenland, yet the rebellion of Gustav Vasa in Sweden and the subsequent end of the Kalmar Union (and Christian II's rule) disrupted those plans. I would think we'd want the Kalmar Union to remain intact for maximal Danish colonisation (much as Scotland was involved in English colonisation, Sweden and Finland might be as well for Denmark), but given Norby's exile from Denmark we'd at least want Christian II to not be deposed in 1523.
What might Denmark be able to settle and hold? Maybe a Caribbean empire as big as the Dutch Caribbean and probably most or all of Canada or at the very least the Maritimes, Quebec, and perhaps most of Ontario. They could establish themselves early in the fur trade and reap large profits as well as make plenty of native allies as a hedge against anyone else who wants to settle in North America.
