It also didn't help that for a long time, the Board of Trade and the Admiralty actively worked to limit settlement in Newfoundland, seeing it largely as nothing more than a seasonal fishery settlement which did not justify the need for a permanent population - which other population groups quickly proved wrong. Now, maybe
@Jürgen is onto something with the Scandinavian experience of switching to dairy farming and all that, which may help, but ultimately the main thing is that to get a larger, more permanent settlement in Newfoundland outside of the fisheries requires squeezing as much life as can be done from the land, with whatever agricultural potential there is, with the seaweed and fish as a start. To get that to work would rely on better relations with the Aboriginal peoples and other non-British population groups.