I believe there was quite a bit of pressure to establish a penal colony somewhere, anywhere. Georgia was the original penal colony, and then Australia once the Colonies were lost. Considering how important East Asia was to the British, and how Australia was comparatively empty, it was a natural choice for settlement. Even if another country controlled it, like France or the Netherlands, I imagine that Britain would attempt to purchase a portion of it, if not seize it outright. Ultimately, I don't think demographics would have changed that much. Without Australia, there still would have been the typical push-pull factors (overpopulation, economic depression, famine) that would have encouraged emigration.