More likely, "fail to gain" - the British Empire grew substantially in the 19th century, especially the later half.
I agree. But let me give it a go and try as hard as I can.
After Napoleon I is defeated and exiled, 1815:
Some form of Federalist party begins to rise in Britain while not rising anywhere else within the empire (or at least no one that matters) and in the first half of the 1800s they set about uniting the british empire into a federal empire.
While Canada, Australia, South Africa and New Zealand are all formed into federal dominions (autonomous at the moment.) These federalist, while helping to build the power of the dominions were also feeding the very heart of each dominions nationality; Meanwhile they keep the empire from expanding.
By the second half of the 1800s the British Federalists are the majority, and the dominionitairy parliments are becoming more and more anti-UK. In the late 1800s a Second English Civil War breaks out with Britain having to fight guerillas across the globe from Canada to India, who (powered by Russia and the US) quickly become conventional armies who combinded provide to be more than the UK can handle and is forced in the end to decolonize at least 50% of its empire, if not in the upwards of 70%.
So maybe 30% of what it was in 1900?