As a person who has lived in Hong Kong in the past, I can tell you that it's not just one city. It is made up of the city itself and various satellite towns. Most of these towns are in the New Territories. Hong Kong is not as small as one might think. And Lantau Island is much bigger than Hong Kong Island itself. You have Kowloon on the NT side of the harbour, and Victoria on HK island itself. Obviously, the Kowloon side is bigger in terms of city sprawl.
On HK island there are smaller towns scattered around it distinct from Victoria. The new airport is built on an artificial island on the northern coast of Lantau Island. The location was chosen since it is sheltered by Lantau and the area is much less built up making landings much easier than the notoriously difficult-to-land-at Death Star style run weaving between skyscrapers at the old airport.
In terms of travel time, it takes about 45 minutes by MTR from the new town of Tung Chung on Lantau near the airport to reach Central Station on HK Island. If you go from Tai-O at the western end of Lantau, you are looking at about almost a 2 hour journey to get to central by bus and then MTR to Central. You would need a significant amount of more time added to get to the south of HK island, or up to the north-east to the eastern extremes of the New Territories, or north to the Chinese border
The MTR goes under the harbour on it's route to HK island, but across bridges (one of them, Tsing Ma is the 6th greatest suspension bridge in the world) linked by the heavily built up Island of Tsing Yi when coming from Lantau.
It is true that HK Island and Kowloon by itself are not viable without the New Territories, and without Lantau too, since the airport is there. Don't forget that reservoirs supplying the entire place with water are in the New Territories and across the border in China as well. Many people have families scattered across the Territory, and jobs that may be on a side opposite from their homes. Splitting HK apart is just not doable, and the British government knew this, which is why the whole lot was handed back, even though HK Island was ceded in perpetuity.
Any POD would probably need to be in the 19th Century with the NT and Lantau as well as smaller islands being ceded in perpetuity as well if you want HK to stay as one successful unit. Unlike Macau, which the Portuguese were not obliged to give up, but voluntarily did so, Hong Kong is a more complex issue.
Sargon