What could've stopped Australia from federating in 1901?

Other than the unlikely scenario of Queen Victoria not giving royal assent to the Constitution of Australia Act, what could've stopped Australia from federating in 1901 and what would've been the result of federation not occurring?
 
I'd think by the time of the Russo-Japan war and Japan's rise to prominence as a pacific power the Australian colonies might get a bit antsy about being a collection of sparsely populated separate colonies and would look to federation as a way for a common defense. Similar to the Canadian colonies around the time of the US civil war.

I think somehow war between the colonies themselves would have been the only thing stopping federation on the continent.
 
A different approach to the metropole's future relationship with the white colonies of settlement could see more direct integration within the UK as an alternative to federation. Different recommendations from the Durham Report as an example for a POD.
 
Less Asian immigration in general, that hugely influenced the colonies federating due to the "Yellow peril" syndrome that swept over Australia, New Zealand, United States, during this period, and mentioned above a less aggressive Japan.
 
Have Canada's federation in 1867 turn to shit, our colonies and Britain won't be so keen then.
 
Top