IOTL they seemingly had a seasonal trade outpost there to trade with the Dorset people, or something similar, some contact occurred. I suppose they sniffed around and decided that a permanent settlement wasn't worth it, which looks like eminently sensible.
There are other better places where you could theoretically eke out a living. Baffin Island is so desolate there's almost nothing that could sustain Medieval peoples.
Ellesmere island would be a better candidate IMHO (and there have been norse artifacts found there so they knew of it). There is open water year around (due to the upwelling of currents) which makes for some of the best (and richest $$$) hunting in the arctic, and there is a fuel source (petrified wood on the neighbouring island to the west, which the Norse also visited and possible utilized IOTL). So we have a reliable food source, a source of income (ivory and hides) and a fuel source.
IMHO its only a matter of time until we find evidence of seasonable occupation of this location IOTL (for we know they were visiting this location for centuries IOTL) never mind this AH challenge. But i degrees...