I know people have already touched upon the options of the Macaronesia (Azores, Madeira, Canary, and Cape Verde Islands), but I looked a bit closer and came across something that made me a bit more interested in these seemingly unimportant colonies.
The Volta do mar. Basically due to currents and wind patterns, it was easy to reach these islands from Iberia but difficult to return from them. The volta do mar is a sailing technique where after reaching these islands, sailors would sail to the to the northwest from these islands to catch eastbound currents back to Iberia and Europe. It was against previous logic as it requires sailors to sail away from home to eventual return there. It basically established a circular sail pattern that utilized the oceanic gyres. It was a major discovery that opened the way to European imperialism as it gave Europe understanding of how to use the trade winds. Portugal utilizing the experience they gained from the the Macaronesia Islands regarding the volta do mar to basically spread throughout the world's oceans largely in thirty years between 1492-1522.
So to reach and colonize these islands would require the Norse figuring out the volta do mar. Now I doubt this would have mattered a great deal with the early Viking/Norse naval vessels. The caravel was basically designed for Portugese exploration down the west African coast, as it was small and maneuverable while the lanteen sails allowed greater tacking. So it isn't like the vikings could have rounded Africa from utilizing the volta do mar, but if nothing else these islands being inhabited could serious screw with the Portugese Age of Discovery, and thus the Spanish one. Portugal wouldn't be settling previously uninhabited islands, and thus gain experience in the volta do mar.
Now this is something of a stretch, but I think this at least a possible series of events that could lead to significantly more Norse colonies. Tell me if you agree or disagree with its plausibility.
Some POD gives the Vikings, as after the Viking Age there weren't really enough Norse trying to settle other places, more success in Iberia. Maybe Vikings make an alliance with Asturias, and their attacks on Lisbon in 844 or 966 is more successful. A more serious migration of Norse follows, and they explore the nearby waters down the northwest African coast. They find the uninhabited Madeira, and maybe launch raids of the Canary Islands. Maybe even capture and settle a few of the smaller ones. The sailing to and fro these places eventually lead the Norse settlers to discover the Azores, and the travel reveals the volta do mar technique to the Norse colonists.
Now the Norse presence in Iberia is short, but no one else has the sailing expertise to reach these now Norse islands. Unless they go completely isolationist though, the Norse would probably need to trade. So either these islands go for the closer Iberian countries, and probably ends up annexed by some point. Or it could try to maintain weak ties to its cultural forebears in Scandinavia. Trade links between these islands and Norway/Denmark develop during and after the end of the Viking Age. Eventually one formally annexes them, I'm going to say Norway as Denmark was more focused on the Baltic. Obviously actual control would be limited, likely depending more on economic costs/benefits than military force. Closer to Greenland than anything.
The Kalmar Union was founded basically to counter the Hanseatic League. It would be hard to overcome the base the Hanseatic League had in the German States, but it might have been possible for the Kalmar Union to nullify their influence by founding trade links entirely separate from those controlled by the Hanseatic Cities. Portugal brought home enough gold from Guinea through their exploration that it expanded their commercial interest, proving the exploration could be profitable. If the independent exploration by the Macaronesia Islands showed a similar profit, it might draw interest. Queen Margaret could choose to sponsor expeditions, and crucially make it a priority to hold onto the islands. The Scandinavian countries had the wood, shipbuilding industries, and naval culture to pull it off. Regular barges had been used by the Portugese for the exploration till the need for lighter and more maneuverable vessels caused them to create the caravel. Lanteen sails were also broadly used in the Mediterranean, so the Norse islands could potentially develop it.
If against all the odds all this happened, the Kalmar Union would be in position to potentially have a Norse Age of Exploration half a century before the historic Portugese one. The Norse could be the ones to establish the trade routes to the Far East/Indian Ocean, not to mention South America and the Caribbean that also followed. I doubt the Kalmar Union would be in the position or have the strength to preempt or emulate the early Portugese and Spanish Empires, and I'm not even going to touch upon the internal politics of the Kalmar Union and how it would effect their exploration efforts, but you might be able to see the Kalmar Union or succeeding Denmark-Norway have major roles in the early colonial empires.
I understand its a very unlikely series of events. The Norse find and colonize Macaronesia>The Macaroneisian colonies maintain closer ties with distant Scandinavia than Iberia>The colonies are annexed by some point by Norway or the Kalmar Union>The entity then chooses to explore down the western coast of Africa. None impossible, but very unlikely all together. Especially as it would require events occurring over centuries going this way, without a clear end goal for the people of the time.
Still interesting idea on a Norse Age of Discovery that doesn't require Vinland or anything. What do you guys think? Unlikely but possible, or outright ASB series of events?